{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "12_Embeddings",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "metadata": {
        "id": "bOChJSNXtC9g",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Embeddings"
      ]
    },
    {
      "metadata": {
        "id": "OLIxEDq6VhvZ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/logo.png\" width=150>\n",
        "\n",
        "So far, we've represented text in a bagged one-hot encoded form which is a n-dimensional array where each index corresponds to a token. The value at that index corresponds to the number of times the word appears in the sentence. This method forces us to completely lose the structural information in our inputs. \n",
        "\n",
        "```python\n",
        "[0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0.\n",
        " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0.\n",
        " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
        " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]```\n",
        " \n",
        " We've also represented our input in a one-hot encoded form where each token is represented by an n-dimensional array. T\n",
        " \n",
        " ```python\n",
        "[[0. 0. 0. ... 0. 0. 0.]\n",
        " [0. 0. 1. ... 0. 0. 0.]\n",
        " [0. 0. 0. ... 0. 0. 0.]\n",
        " ...\n",
        " [0. 0. 0. ... 0. 0. 0.]\n",
        " [0. 0. 0. ... 0. 0. 0.]\n",
        " [0. 0. 0. ... 0. 0. 0.]]\n",
        "```\n",
        "\n",
        "his allows us to preserve the structural information but there are two major disadvantages here. If we have a large vocabulary, the representation length for each token will be massive leading to large computes. And though we preserve the structure within the text, the actual representation for each token does not preserve any relationship with respect to other tokens.\n",
        "\n",
        "In this notebook, we're going to learn about embeddings and how they address all the shortcomings of the representation methods we've seen so far.\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "VoMq0eFRvugb",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Overview"
      ]
    },
    {
      "metadata": {
        "id": "qWro5T5qTJJL",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "* **Objective:**  Represent tokens in text that capture the intrinsic semantic relationships.\n",
        "* **Advantages:** \n",
        "    * Low-dimensionality while capturing relationships.\n",
        "    * Interpretable token representations\n",
        "* **Disadvantages:** None\n",
        "* **Miscellaneous:** There are lot's of pretrained embeddings to choose from but you can also train your own from scratch."
      ]
    },
    {
      "metadata": {
        "id": "2thIKTLYwzJd",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Learning embeddings"
      ]
    },
    {
      "metadata": {
        "id": "MlxioJLqx2Ls",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The main idea of embeddings is to have fixed length representations for the tokens in a text regardless of the number of tokens in the vocabulary. So instead of each token representation having the shape [1XV] where V is vocab size, each token now has the shape [1 X D] where D is the embedding size (usually 50, 100, 200, 300). The numbers in the representation will no longer be 0s and 1s but rather floats that represent that token in a D-dimensional latent space. If the embeddings really did capture the relationship between tokens, then we should be able to inspect this latent space and confirm known relationships (we'll do this soon).\n",
        "\n",
        "But how do we learn the embeddings the first place? The intuition behind embeddings is that the definition of a token depends on the token itself but on it's context. There are several different ways of doing this:\n",
        "\n",
        "1. Given the word in the context, predict the target word (CBOW - continuous bag of words).\n",
        "2. Given the target word, predict the context word (skip-gram).\n",
        "3. Given a sequence of words, predict the next word (LM - language modeling).\n",
        "\n",
        "All of these approaches involve create data to train our model on. Every word in a sentence becomes the target word and the context words are determines by a window. In the image below (skip-gram), the window size is 2. We repeat this for every sentence in our corpus and this results in our training data for the unsupervised task. This in an unsupervised learning technique since we don't have official labels for contexts. The idea is that similar target words will appear with similar contexts and we can learn this relationship by repeatedly training our mode with (context, target) pairs.\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/skipgram.png\" width=600>\n",
        "\n",
        "We can learn embeddings using any of these approaches above and some work better than others. You can inspect the learned embeddings but the best way to choose an approach is to empirically validate the performance on a supervised task. We can learn embeddings by creating our models in PyTorch but instead, we're going to use a library that specializes in embeddings and topic modeling called [Gensim](https://radimrehurek.com/gensim/). "
      ]
    },
    {
      "metadata": {
        "id": "uGDGEVvz41LL",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 734
        },
        "outputId": "90061167-dbee-45f4-986e-e47446a4cd4f"
      },
      "cell_type": "code",
      "source": [
        "!pip install gensim "
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting gensim\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/27/a4/d10c0acc8528d838cda5eede0ee9c784caa598dbf40bd0911ff8d067a7eb/gensim-3.6.0-cp36-cp36m-manylinux1_x86_64.whl (23.6MB)\n",
            "\u001b[K    100% |████████████████████████████████| 23.6MB 1.6MB/s \n",
            "\u001b[?25hCollecting smart-open>=1.2.1 (from gensim)\n",
            "  Downloading https://files.pythonhosted.org/packages/4b/1f/6f27e3682124de63ac97a0a5876da6186de6c19410feab66c1543afab055/smart_open-1.7.1.tar.gz\n",
            "Requirement already satisfied: six>=1.5.0 in /usr/local/lib/python3.6/dist-packages (from gensim) (1.11.0)\n",
            "Requirement already satisfied: scipy>=0.18.1 in /usr/local/lib/python3.6/dist-packages (from gensim) (1.1.0)\n",
            "Requirement already satisfied: numpy>=1.11.3 in /usr/local/lib/python3.6/dist-packages (from gensim) (1.14.6)\n",
            "Collecting boto>=2.32 (from smart-open>=1.2.1->gensim)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl (1.4MB)\n",
            "\u001b[K    100% |████████████████████████████████| 1.4MB 16.0MB/s \n",
            "\u001b[?25hCollecting bz2file (from smart-open>=1.2.1->gensim)\n",
            "  Downloading https://files.pythonhosted.org/packages/61/39/122222b5e85cd41c391b68a99ee296584b2a2d1d233e7ee32b4532384f2d/bz2file-0.98.tar.gz\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.6/dist-packages (from smart-open>=1.2.1->gensim) (2.18.4)\n",
            "Collecting boto3 (from smart-open>=1.2.1->gensim)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/9c/b9/7df67f1775d240ac8d111211f967fa75ecc9968ae79ffa0594e36345445f/boto3-1.9.62-py2.py3-none-any.whl (128kB)\n",
            "\u001b[K    100% |████████████████████████████████| 133kB 27.9MB/s \n",
            "\u001b[?25hRequirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->smart-open>=1.2.1->gensim) (2018.11.29)\n",
            "Requirement already satisfied: urllib3<1.23,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->smart-open>=1.2.1->gensim) (1.22)\n",
            "Requirement already satisfied: idna<2.7,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->smart-open>=1.2.1->gensim) (2.6)\n",
            "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->smart-open>=1.2.1->gensim) (3.0.4)\n",
            "Collecting s3transfer<0.2.0,>=0.1.10 (from boto3->smart-open>=1.2.1->gensim)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/d7/14/2a0004d487464d120c9fb85313a75cd3d71a7506955be458eebfe19a6b1d/s3transfer-0.1.13-py2.py3-none-any.whl (59kB)\n",
            "\u001b[K    100% |████████████████████████████████| 61kB 22.6MB/s \n",
            "\u001b[?25hCollecting botocore<1.13.0,>=1.12.62 (from boto3->smart-open>=1.2.1->gensim)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/e2/77/35e82076e3beb506280f94213a258819378115f174e516ce69b3a2336e1c/botocore-1.12.62-py2.py3-none-any.whl (5.1MB)\n",
            "\u001b[K    100% |████████████████████████████████| 5.1MB 6.3MB/s \n",
            "\u001b[?25hCollecting jmespath<1.0.0,>=0.7.1 (from boto3->smart-open>=1.2.1->gensim)\n",
            "  Downloading https://files.pythonhosted.org/packages/b7/31/05c8d001f7f87f0f07289a5fc0fc3832e9a57f2dbd4d3b0fee70e0d51365/jmespath-0.9.3-py2.py3-none-any.whl\n",
            "Collecting docutils>=0.10 (from botocore<1.13.0,>=1.12.62->boto3->smart-open>=1.2.1->gensim)\n",
            "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/36/fa/08e9e6e0e3cbd1d362c3bbee8d01d0aedb2155c4ac112b19ef3cae8eed8d/docutils-0.14-py3-none-any.whl (543kB)\n",
            "\u001b[K    100% |████████████████████████████████| 552kB 19.3MB/s \n",
            "\u001b[?25hRequirement already satisfied: python-dateutil<3.0.0,>=2.1; python_version >= \"2.7\" in /usr/local/lib/python3.6/dist-packages (from botocore<1.13.0,>=1.12.62->boto3->smart-open>=1.2.1->gensim) (2.5.3)\n",
            "Building wheels for collected packages: smart-open, bz2file\n",
            "  Running setup.py bdist_wheel for smart-open ... \u001b[?25l-\b \b\\\b \bdone\n",
            "\u001b[?25h  Stored in directory: /root/.cache/pip/wheels/23/00/44/e5b939f7a80c04e32297dbd6d96fa3065af89ecf57e2b5f89f\n",
            "  Running setup.py bdist_wheel for bz2file ... \u001b[?25l-\b \bdone\n",
            "\u001b[?25h  Stored in directory: /root/.cache/pip/wheels/81/75/d6/e1317bf09bf1af5a30befc2a007869fa6e1f516b8f7c591cb9\n",
            "Successfully built smart-open bz2file\n",
            "Installing collected packages: boto, bz2file, docutils, jmespath, botocore, s3transfer, boto3, smart-open, gensim\n",
            "Successfully installed boto-2.49.0 boto3-1.9.62 botocore-1.12.62 bz2file-0.98 docutils-0.14 gensim-3.6.0 jmespath-0.9.3 s3transfer-0.1.13 smart-open-1.7.1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "9wr9S6965DD7",
        "colab_type": "code",
        "outputId": "78009f70-4ae5-436e-b4e9-065ff5e9a86b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "import os\n",
        "from argparse import Namespace\n",
        "import copy\n",
        "import gensim\n",
        "from gensim.models import Word2Vec\n",
        "import json\n",
        "import nltk; nltk.download('punkt')\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import re\n",
        "import urllib\n",
        "import warnings\n",
        "warnings.filterwarnings('ignore')"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Unzipping tokenizers/punkt.zip.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "5-sx-n9655TJ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "args = Namespace(\n",
        "    seed=1234,\n",
        "    data_file=\"harrypotter.txt\",\n",
        "    embedding_dim=100,\n",
        "    window=5,\n",
        "    min_count=3,\n",
        "    skip_gram=1, # 0 = CBOW\n",
        "    negative_sampling=20,\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "VaJSRgNEg_V_",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Upload data from GitHub to notebook's local drive\n",
        "url = \"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/data/harrypotter.txt\"\n",
        "response = urllib.request.urlopen(url)\n",
        "html = response.read()\n",
        "with open(args.data_file, 'wb') as fp:\n",
        "    fp.write(html)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "irvgngOG5yqk",
        "colab_type": "code",
        "outputId": "086fa4d4-57d3-42fe-ba5e-6f15d1fa518a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "# Split text into sentences\n",
        "tokenizer = nltk.data.load('tokenizers/punkt/english.pickle')\n",
        "with open(args.data_file, encoding='cp1252') as fp:\n",
        "    book = fp.read()\n",
        "sentences = tokenizer.tokenize(book)\n",
        "print (len(sentences))\n",
        "print (sentences[11])"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "15640\n",
            "Snape nodded, but did not elaborate.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "VTREFDg47Vrx",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Preprocessing\n",
        "def preprocess_text(text):\n",
        "    text = ' '.join(word.lower() for word in text.split(\" \"))\n",
        "    text = re.sub(r\"([.,!?])\", r\" \\1 \", text)\n",
        "    text = re.sub(r\"[^a-zA-Z.,!?]+\", r\" \", text)\n",
        "    text = text.strip()\n",
        "    return text"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "IX7pkCbg7WPK",
        "colab_type": "code",
        "outputId": "95f9759f-6add-42c6-d4ab-62fcb7fbb58d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Clean sentences\n",
        "sentences = [preprocess_text(sentence) for sentence in sentences]\n",
        "print (sentences[11])"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "snape nodded , but did not elaborate .\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "dsmEEVzX5ytO",
        "colab_type": "code",
        "outputId": "ba5cf8a6-106d-49ed-a9f8-063e04d58eeb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Process sentences for gensim\n",
        "sentences = [sentence.split(\" \") for sentence in sentences]\n",
        "print (sentences[11])"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['snape', 'nodded', ',', 'but', 'did', 'not', 'elaborate', '.']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "Fa-DbjPW-KC3",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "When we have large vocabularies to learn embeddings for, things can get complex very quickly. Recall that the backpropagation with softmax updates both the correct and incorrect class weights. This becomes a massive computation for every backwas pass we do so a workaround is to use [negative sampling](http://mccormickml.com/2017/01/11/word2vec-tutorial-part-2-negative-sampling/) which only updates the correct class and a few arbitrary incorrect classes (negative_sampling=20). We're able to do this because of the large amount of training data where we'll see the same word as the target class multiple times.\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "wzkUkl-b5ywC",
        "colab_type": "code",
        "outputId": "94d29f13-c47e-4c2c-915d-a4d930531346",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Super fast because of optimized C code under the hood\n",
        "model = Word2Vec(sentences=sentences, size=args.embedding_dim, \n",
        "                 window=args.window, min_count=args.min_count, \n",
        "                 sg=args.skip_gram, negative=args.negative_sampling)\n",
        "print (model)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Word2Vec(vocab=4837, size=100, alpha=0.025)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "gcMb9HUd7vS_",
        "colab_type": "code",
        "outputId": "6be3ac88-a5c5-4614-8269-6c0547a6303d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 374
        }
      },
      "cell_type": "code",
      "source": [
        "# Vector for each word\n",
        "model.wv.get_vector(\"potter\")"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([-0.2489659 ,  0.24121806,  0.15966399, -0.1747967 ,  0.10363109,\n",
              "       -0.1511645 ,  0.15248442, -0.04558881, -0.08044929,  0.2222574 ,\n",
              "       -0.25447345,  0.27639526, -0.16762276, -0.14328802, -0.01826264,\n",
              "        0.23642407, -0.05720777,  0.0143759 , -0.03353882,  0.12028241,\n",
              "       -0.01040754,  0.18283844, -0.17171031,  0.31479293,  0.05503249,\n",
              "       -0.08049146,  0.20224565, -0.11427342,  0.05925621,  0.0216365 ,\n",
              "        0.11731622, -0.12440909,  0.3802724 ,  0.1682069 ,  0.50465703,\n",
              "        0.3332439 , -0.4917376 ,  0.20792836,  0.4224515 , -0.33049974,\n",
              "        0.25798362, -0.03507125,  0.21839641,  0.08492662, -0.26844618,\n",
              "        0.07765336,  0.30522913,  0.01970481,  0.13059294, -0.6628051 ,\n",
              "       -0.34673738,  0.479213  ,  0.19385533, -0.14426081, -0.23008673,\n",
              "       -0.20338146, -0.14760706,  0.545037  ,  0.0767005 ,  0.07072604,\n",
              "       -0.26795644,  0.2091712 ,  0.16069579,  0.2414927 , -0.2661651 ,\n",
              "       -0.7027013 , -0.06159785,  0.11271702,  0.18699864,  0.06832824,\n",
              "        0.04442952,  0.261182  , -0.24468403, -0.16012008, -0.02954605,\n",
              "       -0.19371727,  0.4338561 , -0.02107797,  0.36587957, -0.6298013 ,\n",
              "        0.37707213, -0.0692855 ,  0.29460624,  0.03771852,  0.27480027,\n",
              "        0.53056574,  0.00974346,  0.13482095,  0.16975024, -0.32460022,\n",
              "        0.03467605,  0.31499952, -0.05332879,  0.28996226,  0.38008863,\n",
              "       -0.37839958, -0.44602472, -0.10334857,  0.17595556, -0.02492724],\n",
              "      dtype=float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "metadata": {
        "id": "BozlP5WG70Ak",
        "colab_type": "code",
        "outputId": "a82275b3-8d43-41ea-89ec-7be3ccdae930",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "cell_type": "code",
      "source": [
        "# Get nearest neighbors (excluding itself)\n",
        "model.wv.most_similar(positive=\"scar\", topn=5)"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[('pain', 0.9486030340194702),\n",
              " ('burned', 0.9303449988365173),\n",
              " ('forehead', 0.9295971393585205),\n",
              " ('prickling', 0.9275624752044678),\n",
              " ('burning', 0.9198344349861145)]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "metadata": {
        "id": "q-wgfMnH68vQ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Save the weights \n",
        "model.wv.save_word2vec_format('model.txt', binary=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "K1OC2smgw1P5",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Pretrained embeddings"
      ]
    },
    {
      "metadata": {
        "id": "DHUU_-TJz_7K",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "We can learn embeddings from scratch using one of the approaches above but we can also leverage pretrained embeddings that have been trained on millions of documents. Popular ones include Word2Vec (skip-gram) or GloVe (global word-word co-occurrence). We can validate that these embeddings captured meaningful semantic relationships by confirming them."
      ]
    },
    {
      "metadata": {
        "id": "vJNngmZjglVg",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from gensim.scripts.glove2word2vec import glove2word2vec\n",
        "from gensim.models import KeyedVectors\n",
        "from io import BytesIO\n",
        "import matplotlib.pyplot as plt\n",
        "from sklearn.decomposition import PCA\n",
        "from zipfile import ZipFile\n",
        "from urllib.request import urlopen"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "5WZ2bT27gncI",
        "colab_type": "code",
        "outputId": "889cf713-9164-44ae-d004-c393564927f7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "# Unzip the file (may take ~3 minutes)\n",
        "resp = urlopen('http://nlp.stanford.edu/data/glove.6B.zip')\n",
        "zipfile = ZipFile(BytesIO(resp.read()))\n",
        "zipfile.namelist()"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['glove.6B.50d.txt',\n",
              " 'glove.6B.100d.txt',\n",
              " 'glove.6B.200d.txt',\n",
              " 'glove.6B.300d.txt']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "metadata": {
        "id": "NNacKuRNi8_x",
        "colab_type": "code",
        "outputId": "5180e767-57c3-49c8-b96b-503162bec86b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Write embeddings\n",
        "embeddings_file = 'glove.6B.{0}d.txt'.format(args.embedding_dim)\n",
        "zipfile.extract(embeddings_file)"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'/content/glove.6B.100d.txt'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "metadata": {
        "id": "SA5Y1BGbbpPo",
        "colab_type": "code",
        "outputId": "bf3bd287-47bf-442e-c78b-9ec7eee5f70c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Save GloVe embeddings to local directory in word2vec format\n",
        "word2vec_output_file = '{0}.word2vec'.format(embeddings_file)\n",
        "glove2word2vec(embeddings_file, word2vec_output_file)"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(400000, 100)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "metadata": {
        "id": "bQp1pIJLiZw3",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Load embeddings (may take a minute)\n",
        "glove = KeyedVectors.load_word2vec_format(word2vec_output_file, binary=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "HDpHnC0sbpVE",
        "colab_type": "code",
        "outputId": "a19ed390-7c15-4897-d440-65137d170525",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "cell_type": "code",
      "source": [
        "# (king - man) + woman = ?\n",
        "glove.most_similar(positive=['woman', 'king'], negative=['man'], topn=5)"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[('queen', 0.7698541283607483),\n",
              " ('monarch', 0.6843380928039551),\n",
              " ('throne', 0.6755735874176025),\n",
              " ('daughter', 0.6594556570053101),\n",
              " ('princess', 0.6520534753799438)]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "metadata": {
        "id": "_1TatbmvbpYU",
        "colab_type": "code",
        "outputId": "ec827143-0c7f-4c1b-f080-4d1458f53c1e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "cell_type": "code",
      "source": [
        "# Get nearest neighbors (exlcusing itself)\n",
        "glove.wv.most_similar(positive=\"goku\", topn=5)"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[('gohan', 0.7246542572975159),\n",
              " ('bulma', 0.6497020125389099),\n",
              " ('raistlin', 0.6443604230880737),\n",
              " ('skaar', 0.6316742897033691),\n",
              " ('guybrush', 0.6231324672698975)]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "metadata": {
        "id": "oMljB5MEbpdb",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Reduce dimensionality for plotting\n",
        "X = glove[glove.wv.vocab]\n",
        "pca = PCA(n_components=2)\n",
        "pca_results = pca.fit_transform(X)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Aok5pRqXqcCf",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def plot_embeddings(words, embeddings, pca_results):\n",
        "    for word in words:\n",
        "        index = embeddings.index2word.index(word)\n",
        "        plt.scatter(pca_results[index, 0], pca_results[index, 1])\n",
        "        plt.annotate(word, xy=(pca_results[index, 0], pca_results[index, 1]))\n",
        "    plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "NOdLFgTOrrBd",
        "colab_type": "code",
        "outputId": "3ec4bd22-6c24-46c3-889b-0dceda5d536e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 347
        }
      },
      "cell_type": "code",
      "source": [
        "plot_embeddings(words=[\"king\", \"queen\", \"man\", \"woman\"], embeddings=glove, \n",
        "                pca_results=pca_results)"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeQAAAFKCAYAAADMuCxnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHqNJREFUeJzt3Xt0VPW99/HPXHIhTAyTZCKYhAWC\ngILSUPQUoaJpsFR4bLVIQlG897CqrTd6CsFDVORaQRfoozZeagUJLY0o2mNaJCiHS0EFJPAoJBbk\nEpMJuciQBEgyzx+tUynJTGqSmV9m3q+/MrP3zO87n5U1n9l752Lxer1eAQCAkLKGegAAAEAhAwBg\nBAoZAAADUMgAABiAQgYAwAAUMgAABrAHe0G3+0Swl/RxOuNUU1MfsvW7AzLyj3wCIyP/yCewcMzI\n5YoPuE9EHSHb7bZQj2A8MvKPfAIjI//IJ7BIzSiiChkAAFNRyAAAGIBCBgDAABQyAAAGoJABADAA\nhQwAgAEoZPj10UcfKDv7R2fdt29fiR588N4QTQQA4YlCxr/tkkuGaenSp0M9BgCElaD/pS50X01N\nTXrggXt05ZXf1dq1a7R69Vq9+OLzqqurldvtVmnpAfXqlaAFC5YqOTlZn376ifLyZkmSrr32B3rv\nvQ26774ZGjFiZIhfCQCYhyNktNtTT/1a6el9NXjwkLPuLy5+V/fd95B+//u1cjoT9fbbb0iSFi+e\np+zsqSooeF0Oh0OHD38eirEBoFugkNEur7++RkeOHNaDD/7qnG3Dh2eod+8+slgsuuiiwaqo+EKn\nTjXq00//n7Kyvi9JuvHGyfJ6vcEeGwC6DQoZbTp1plm1nlM6fvy4nntuuZKSkmS3n3uVw+Fw+L62\nWq1qaWnRiRMnZLFYFB//9z+obrfb5XQmBm12AOhuuIaMczS3tGj1hlLt3O/Wkc/2qclr1U13L9CG\nPyzWe+8V+0rWn7i4nvJ6vWpsbFRsbKyamppUW1sThOkBoHviCBnnWL2hVOs/OKLjX56SV5LFFqu/\nljbq8mvv0NKlC1VbWxvwOeLi4tSvX39t2PAXSdIbbxRKsnTt4ADQjVHIOEvj6Sbt3O9udVvFqWRd\nfc21WrJkQbue68EHf6Xf/e4l3XzzZDU2NsjlcslioZQBoDUWb5B/0sbtPhHM5c7icsWHdP3uoMli\n1X8uWK/WvimsFmn+T7+jFGdcu5/P6/X6SnjixCw9+eT/1UUXDeqkaYOP76HAyMg/8gksHDNyuQJf\n6mvXEfL+/fuVlZWlFStWtLnPkiVLdMstt7R/OhjJeV6MEs+LaX1bfKwSHK1va83DD/9KK1e+Ikn6\n8MMd8nq96tu3b6fMCQDhJuAPddXX12vu3LkaNWpUm/uUlpZqx44dioqK6tThEHyx0XZlDHJp/QdH\nztmWMShZMVG2dj/XXXdN1/z5j+qtt95UVJRdDz/8mGJiYjtzXAAIGwELOTo6Wvn5+crPz29zn4UL\nF+qBBx7Q00/z5xTDQXbmQEnSzv1VqjnRKGd8rDIGJfvub69+/frrN7/5bRdMCADhJ2Ah2+32Vn/3\n9CuFhYW64oorlJqa2qmDIXRsVqt+kjVIPx47QHWeU0pwxPxbR8YAgH9fh34Puba2VoWFhXr55ZdV\nUVHRrsc4nXGy20P35t6eC+uR7usZpYVwDlPxPRQYGflHPoFFYkYdKuRt27apurpaU6dO1enTp/X5\n559r/vz5ys3NbfMxNTX1HVmyQ8LxJ/c6Gxn5Rz6BkZF/5BNYOGbUng8YHSrk8ePHa/z48ZKkI0eO\naNasWX7LGAAAtC5gIZeUlGjRokU6evSo7Ha7ioqKlJmZqbS0NI0bNy4YMwIAEPb4wyA4Cxn5Rz6B\nkZF/5BNYOGbUaX8YBAAAdC0KGQAAA1DIAAAYgEIGAMAAFDIAAAagkAEAMACFDACAAShkAAAMQCED\nAGAAChkAAANQyAAAGIBCBgDAABQyAAAGoJABADAAhQwAgAEoZAAADEAhAwBgAAoZAAADUMgAABiA\nQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxAIQMAYAAKGQAAA1DIAAAYgEIGAMAAFDIAAAagkAEA\nMACFDACAAShkAAAMQCEDAGAAChkAAANQyAAAGIBCBgDAABQyAAAGoJABADAAhQwAgAEoZAAADEAh\nAwBgAAoZAAADUMgAABiAQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxAIQMAYAAKGQAAA1DIAAAY\noF2FvH//fmVlZWnFihXnbNu2bZsmT56snJwczZo1Sy0tLZ0+JAAA4S5gIdfX12vu3LkaNWpUq9vn\nzJmjZcuWqaCgQCdPntSmTZs6fUgAAMJdwEKOjo5Wfn6+UlJSWt1eWFio3r17S5ISExNVU1PTuRMC\nABABAhay3W5XbGxsm9sdDockqbKyUps3b9bYsWM7bzoAACKEvTOe5Pjx45o+fbry8vLkdDr97ut0\nxslut3XGst+IyxUfsrW7CzLyj3wCIyP/yCewSMyow4Xs8Xh099136/7779eYMWMC7l9TU9/RJb8x\nlytebveJkK3fHZCRf+QTGBn5Rz6BhWNG7fmA0eFfe1q4cKFuvfVWXXXVVR19KgAAIlbAI+SSkhIt\nWrRIR48eld1uV1FRkTIzM5WWlqYxY8Zo7dq1OnTokNasWSNJmjhxorKzs7t8cAAAwknAQh42bJhe\nffXVNreXlJR06kAAAEQi/lIXAAAGoJABADAAhQwAgAEoZAAADEAhAwBgAAoZAAADUMgAABiAQgYA\nwAAUMgAABqCQAQAwAIUMAIABKGQAAAxAIQMAYAAKGQAAA1DIAAAYgEIGAMAAFDIAAAagkAEAMACF\nDACAAShkAAAMQCEDAGAAChkAAANQyAAAGIBCBgDAABQyAAAGoJABADAAhQwAgAEoZAAADEAhAwBg\nAHuoBwAAwCTl5cc0ffrtmjz5J3rrrTfk9UoPP/yoXnnlBR04sF9XXPEd5ebmad26tSooWKHm5mYl\nJSXrv//7MfXu3Ud/+tM6bdnyv+rZs6d2794lu92mp59erosuusjvuhwhAwDwL2pra5WYmKRVqwo1\ncOBA5eXN0uzZj+qVV1Zp/foi7d1boiefXKwnn3xGBQWvKzU1Tb/97Qu+x2/btlk33HCTCgoKlZEx\nUq+88krANSlkAAD+RXNzszIzsyRJF144UBdffIl69eqlhIReSkpKVlPTGRUVvaeUlPMlScOHZ+jY\nsaO+x/frd6GGDLlYkjR48GCVl5cHXJNT1gAA/EPLqVM6c/y4bFabYmJiJUlWq1U9esT59rFarTpz\n5oxeeOE5bd78vpqbm1VfX6/09L6+fXr2dHxtf5uam5sDrk0hAwAinre5We4/FMiz8yN98cUX8rY0\nq7JgpVw35bS6f3V1tTZvfl9PP52vXr166c03X9ef//w/HZqBQgYARDz3HwpUu/4vf7/h9UrSP2+3\nora2Wr1791GvXr1UV1erDRv+ooaGhg7NwDVkAEBEazl1Sp6dH7W6zbNzp7ytnG5OSkpWXV2dsrN/\npEcema277/6ZKisrtHz5k994DovX+4+PAkHidp8I5nJncbniQ7p+d0BG/pFPYGTkH/kEFuyMTldW\n6uDsX/mOjM9itarf4wsVnZLSoTVcrviA+3CEDACIaPaEBNkTE1vf5kyUPSEhKHNQyACAiGaNiZEj\nY0Sr2xwZGbLGxARlDn6oCwAQ8b76aWrPzp1qqqmW3ZkoR0ZGmz9l3RUoZABAxLPYbErJmarkGyap\nqa5O9oSEoB0Zf4VCBgDgH6wxMR3+Aa5vvHZIVgUAAGehkAEAMACFDACAAShkAAAMQCEDAGAAChkA\nAANQyAAAGIBCBgDAABQyAAAGaFch79+/X1lZWVqxYsU527Zs2aJJkyYpOztbzzzzTKcPCABAJAhY\nyPX19Zo7d65GjRrV6vbHH39cy5cv16pVq7R582aVlpZ2+pAAAIS7gIUcHR2t/Px8pbTytz0PHz6s\nhIQE9enTR1arVWPHjtXWrVu7ZFAAAMJZwEK22+2KjY1tdZvb7Vbi1/6pc2Jiotxud+dNBwBAhAj6\nf3tyOuNkt9uCvayPyxUfsrW7CzLyj3wCIyP/yCewSMyoQ4WckpKiqqoq3+2KiopWT21/XU1NfUeW\n7BCXK15u94mQrd8dkJF/5BMYGflHPoGFY0bt+YDRoV97SktLk8fj0ZEjR9TU1KTi4mKNHj26I08J\nAEBECniEXFJSokWLFuno0aOy2+0qKipSZmam0tLSNG7cOD3yyCN66KGHJEnXXXed+vfv3+VDAwAQ\nbixer9cbzAVDeRoiHE+DdDYy8o98AiMj/8gnsHDMqMtPWQMAgM5BIQMAYAAKGQAAA1DIAAAYgEIG\nAMAAFDIAAAagkAEAMACFDACAAShkAAAMQCEDAGAAChkAAANQyAAAGIBCBgDAABQyAAAGoJABADAA\nhQwAgAEoZAAADEAhAwBgAAoZAAADUMgAABiAQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxAIQMA\nYAAKGQAAA1DIAAAYgEIGAMAAFDIAAAagkAEAMACFDACAAShkAAAMQCEDAGAAChkAAANQyAAAGIBC\nBgDAABQyAAAGoJABADAAhQwAgAEoZAAADEAhAwBgAAoZAAADUMgAABiAQgYAwAAUMgAABqCQAQAw\nAIUMAIABKGQAAAxAIQMAYAAKGQAAA9jbs9P8+fO1e/duWSwW5ebm6rLLLvNtW7lypd58801ZrVYN\nGzZMs2fP7rJhAQAIVwGPkLdv365Dhw5p9erVmjdvnubNm+fb5vF49OKLL2rlypVatWqVysrKtGvX\nri4dGACAcBSwkLdu3aqsrCxJ0oABA1RXVyePxyNJioqKUlRUlOrr69XU1KSGhgYlJCR07cQAAISh\ngKesq6qqNHToUN/txMREud1uORwOxcTE6J577lFWVpZiYmI0YcIE9e/f3+/zOZ1xstttHZ/8G3K5\n4kO2dndBRv6RT2Bk5B/5BBaJGbXrGvLXeb1e39cej0fPP/+83nnnHTkcDt1666365JNPNGTIkDYf\nX1NT/80m7QQuV7zc7hMhW787ICP/yCcwMvKPfAILx4za8wEj4CnrlJQUVVVV+W5XVlbK5XJJksrK\nypSenq7ExERFR0dr5MiRKikp6cDIAABEpoCFPHr0aBUVFUmS9u7dq5SUFDkcDklSamqqysrK1NjY\nKEkqKSlRv379um5aAADCVMBT1iNGjNDQoUOVk5Mji8WivLw8FRYWKj4+XuPGjdOdd96padOmyWaz\nKSMjQyNHjgzG3AAAhBWL9+sXhYMglNcFwvG6RGcjI//IJzAy8o98AgvHjDrlGjIAAOh6FDIAAAag\nkAEAMACFDACAAShkAAAMQCEDAGAAChkAAANQyAAAGIBCBgDAABQyAAAGoJABADAAhQwAgAEoZAAA\nDEAhAwBgAAoZAAADUMgAABiAQgYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxAIQMAYAAKGQAAA9hD\nPQCA8HfjjRO0bNlzSktL17vv/llz587RO+9sVGxsrAoKVujQoUOyWi366KMPZLVa9Z3vjNbPfvYL\n2Ww2TZr0f5STM1V/+tM6ud1uzZgxUx98sEN//esW9erl1BNPLNN5552nkpKPtXTpYjU2Nshqteq+\n+2bo8sv/Q+XlxzR9+u26+ebbtW7d6/ryyy/1858/oO9979pQxwKchSNkAF0uI+PbKin5WJK0a9dO\nDR58sfbtK5Ek7d69Sy6XS5WVFXr11d/rpZdW6OOPd2r9+iLf4z/7rEwvvbRSt912p+bOnaNrrvme\nVq9eK6+3Re+/v0GStHjxPP3kJ7fotdf+qKlTb9UTTyzwPb62tlZWq0W/+91q/eIXDyk//9kgvnqg\nfShkAF3mzJlm1dU0aPjwESop2SNJ2rt3jyZO/KH27NktSdq3b482bdqo66+/QXa7XTExsRo37gfa\nvn2b73m++92rJUkXXjhQMTExGjFipCwWi/r3H6CqqipJ0ssvv6bMzHGSpOHDM3Ts2FHf45ubm3Xd\ndddLkgYPHqKKii+6/LUD/y5OWQPodC0tLdqyoUx/218lz5en5LXa9dftO1RXV6uoqCiNGDFSTz65\nWIcOHVRKSm+dPOlRfPx5vsfHx8erpqbGdzsurqckyWq1qUePON/9VqtVzc3NkqQ///l/tGbNatXX\nn1RLS4u8Xq9vP5vNph49evge09LS0qWvH/gmKGQAnW7LhjLt+eCfR6iWFoc8X57Uqy+t1bBhlyo1\nNU3l5ce0e/dOjRx5hfbs2a26ujrf/l9+WafExMR2r+d2V2rx4nn6zW9+q4suGqzDhz/XlCk3dupr\nAroap6wBdKozp5v0t/1V59zvSuqvDRvf0sUXXypJ6tu3n95++019+9uX68orx+jtt99Qc3OzGhoa\nVFT0J40aNabda9bW1ig2tof69u2npqYmvfnm65Kk+vr6znlRQBBQyAA61YkvT8nz5alz7j8/aaAq\njx/UgP5DJEmXXnqZDhz4VJdeOlw//nG2UlLO1y23TNZdd92iK6/8rjIzs9q95sCBgzRq1GhNmXKj\npk+/Q6NHf1dDh16qe+/9aae9LqCrWbxfv9ASBG73iWAudxaXKz6k63cHZOQf+QTWK6GHli/Y0Gop\nx58Xo+y7r1BUlC0Ek5mB76HAwjEjlys+4D4cIQPoVFHRdvUflNzqtn6DkiO6jAF/+KEuAJ3uyswB\nkqSD+6vkOXFKjvgY9RuU7LsfwLkoZACdzmq1akzWRfqPsReq3nNacY5ojoyBAChkAF0mKsqmBGeP\nUI8BdAtcQwYAwAAUMgAABqCQAQAwAIUMAIABKGQAAAxAIQMAYAAKGQAAA1DIAAAYgEIGAMAAFDIA\nAAagkAEAMACFDACAAShkAAAMQCEDAGAAChkAAANQyAAAGMDenp3mz5+v3bt3y2KxKDc3V5dddplv\nW3l5uR588EGdOXNGl1xyiR577LEuGxYAgHAV8Ah5+/btOnTokFavXq158+Zp3rx5Z21fuHCh7rjj\nDq1Zs0Y2m03Hjh3rsmEBAAhXAQt569atysrKkiQNGDBAdXV18ng8kqSWlhZ9+OGHyszMlCTl5eXp\nggsu6MJxAQAITwFPWVdVVWno0KG+24mJiXK73XI4HKqurlbPnj21YMEC7d27VyNHjtRDDz3k9/mc\nzjjZ7baOT/4NuVzxIVu7uyAj/8gnMDLyj3wCi8SM2nUN+eu8Xu9ZX1dUVGjatGlKTU3VT3/6U23c\nuFFXX311m4+vqan/RoN2BpcrXm73iZCt3x2QkX/kExgZ+Uc+gYVjRu35gBHwlHVKSoqqqqp8tysr\nK+VyuSRJTqdTF1xwgfr27SubzaZRo0bpwIEDHRgZAIDIFLCQR48eraKiIknS3r17lZKSIofDIUmy\n2+1KT0/XwYMHfdv79+/fddMCABCmAp6yHjFihIYOHaqcnBxZLBbl5eWpsLBQ8fHxGjdunHJzczVz\n5kx5vV4NGjTI9wNeAACg/Szer18UDoJQXhcIx+sSnY2M/COfwMjIP/IJLBwz6pRryAAAoOtRyAAA\nGIBCBgDAABQyAAAGoJABADAAhQwAgAEoZAAADEAhAwBgAAoZAAADUMgAABiAQgYAwAD/9v9DNtXL\nL+dr3bq1SkhIUGbmtXrjjT8qI+PbSk1N02233SVJmjlzppKSztdtt92lv/3tMy1ZslBVVVWKjo5S\nbm6ehgy5RJL0xhuFWr16pU6fPq2hQy9Vbu4cxcTEat68R9S7dx/t2bNbhw9/rvT0vlq4cKliY2ND\n+dIBAGEgLI6QP/usVKtXv6b8/FeUn/877du3x+/+LS0tmjVrhsaPv04FBYWaMWOWZs58SE1NTdq9\ne6deeOE5LVv2nNasWSeHw6H8/Od8jy0uXq/HHlug1avXqra2Vu+/X9zVLw8AEAHCopA//niXMjJG\nKCkpWXa7XVlZ4/3uf+jQQdXWVmvChB9Kki677Fvq1cupkpKPtXnz+/re98YpOdklSfrRj36s9977\nZ+mOGjVG552XILvdrgEDBqii4ouue2EAgIjR7U9Zn24+rfLjXyiuZ0/ffU6n0+9jPJ4Tamxs1NSp\nk3z3nTx5UnV1dTpxwqNNm4q1ffs2SVJLi1dNTWd8+/X82jpWq03Nzc2d9VIAABGs2xZyc0uzCkvf\n1sfuvTrgLlHj4RP6w/43dePACaqtrZUkWa1WtbS0+B5TV1enpKTzlZzsUs+ePfXaa38853lLS/dr\n/PiJuvfe+4P2WgAA6LanrAtL39bGI/+r6lM16pF+nmr+5tb6Tzdozafr9M47b0mSkpKSVVp6QJJ0\n9OgRffTRR5Kk3r37yOU6X8XF6yVJtbW1ysvLVUNDg8aMGav33itWTU2NJGnTpo1aseK3wX+BAICI\n0i2PkE83n9bH7r2+23F94pX07Qu0/9kdOuTYo5t+MFmffVam66+/Qbm5M5STc4MGDRqi73//+5Ik\ni8WiRx+dr1//er7y85+V1WpVdvZU9ejRQ4MHD9G0abfr5z//T3m9LXI6E/XLX+aG6qUCACKExev1\neoO5oNt9ouPPUX9cj25bLK/OHd0ii25yXKfli5/QmjXrztrmcsV3yvrhjIz8I5/AyMg/8gksHDNy\nueID7tMtT1knxMTLGdOr1W2Jsb3kiIoL8kQAAHRMtyzkaFu0LnMNbXXbpclDZbdGBXkiAAA6plte\nQ5akGwdOkCTtqdqr6sZaJcb20qXJQ3XjwAmyWW3nnK4GAMBk3baQbVabbhp0vX44YLzqTp1QQky8\nom3RoR4LAIBvpNsW8leibdFyxSWFegwAADqkW15DBgAg3FDIAAAYgEIGAMAAFDIAAAagkAEAMACF\nDACAAShkAAAMQCEDAGCAoP+3JwAAcC6OkAEAMACFDACAAShkAAAMQCEDAGAAChkAAANQyAAAGCDs\nCrmhoUH33Xefbr75Zt10000qLi4+a3t5ebmmTJmiSZMmac6cOSGaMrQCZbRy5UplZ2drypQpmjdv\nXoimDL3GxkZlZWWpsLDwrPu3bNmiSZMmKTs7W88880yIpjNDWxlt27ZNkydPVk5OjmbNmqWWlpYQ\nTRhabeXzlSVLluiWW24J8lRmaSujSHyvDrtCLi4u1rBhw7RixQo99dRTWrhw4VnbFy5cqDvuuENr\n1qyRzWbTsWPHQjRp6PjLyOPx6MUXX9TKlSu1atUqlZWVadeuXSGcNnSeffZZJSQknHP/448/ruXL\nl2vVqlXavHmzSktLQzCdGdrKaM6cOVq2bJkKCgp08uRJbdq0KQTThV5b+UhSaWmpduzYEeSJzNNW\nRpH4Xm0P9QCd7brrrvN9XV5ervPPP993u6WlRR9++KGWLl0qScrLywv6fCbwl1FUVJSioqJUX1+v\nuLg4NTQ0tPmGEs7KyspUWlqqq6+++qz7Dx8+rISEBPXp00eSNHbsWG3dulUDBw4MwZSh1VZGklRY\nWCiHwyFJSkxMVE1NTZCnCz1/+Uh/L5wHHnhATz/9dHAHM0hbGUXqe3XYHSF/JScnRzNmzFBubq7v\nvurqavXs2VMLFizQlClTtGTJkhBOGHqtZRQTE6N77rlHWVlZuuaaazR8+HD1798/hFOGxqJFizRz\n5sxz7ne73UpMTPTdTkxMlNvtDuZoxmgrI0m+Mq6srNTmzZs1duzYYI5mBH/5FBYW6oorrlBqamqQ\npzJLWxlF6nt12BZyQUGBnn32Wf3yl7/UV38d1Ov1qqKiQtOmTdOKFSu0b98+bdy4MbSDhlBrGXk8\nHj3//PN655139O6772r37t365JNPQjxpcK1du1bf+ta3lJ6eHupRjNWejI4fP67p06crLy9PTqcz\niNOFnr98amtrVVhYqNtvvz0Ek5nDX0aR+l4ddqesS0pKlJSUpD59+ujiiy9Wc3OzqqurlZSUJKfT\nqQsuuEB9+/aVJI0aNUoHDhxo85RSuPKXUVlZmdLT031HgSNHjlRJSYmGDBkS4qmDZ+PGjTp8+LA2\nbtyoL774QtHR0erdu7euvPJKpaSkqKqqyrdvRUWFUlJSQjhtaPjLSPr7B7u7775b999/v8aMGRPi\naYPPXz7btm1TdXW1pk6dqtOnT+vzzz/X/PnzzzpTFQn8ZRSp79VhV8gffPCBjh49qtmzZ6uqqkr1\n9fW+T+d2u13p6ek6ePCg+vXrp71792rChAkhnjj4/GWUmpqqsrIyNTY2KjY2ViUlJRF3uvGpp57y\nfb18+XKlpqb6iiYtLU0ej0dHjhxR7969VVxcrCeeeCJUo4aMv4ykv18fvfXWW3XVVVeFYryQ85fP\n+PHjNX78eEnSkSNHNGvWrIgrY8l/RpH6Xh12/+2psbFRs2fPVnl5uRobG3XvvfeqtrZW8fHxGjdu\nnA4dOqSZM2fK6/Vq0KBBeuSRR2S1hu2Z+1YFyqigoECFhYWy2WzKyMjQf/3Xf4V65JD56o1Cki+f\nHTt2+Er42muv1Z133hnKEUPuXzMaM2aMLr/8cmVkZPj2mThxorKzs0M1Yki19j30la8K+dVXXw3V\neEZoLaNIfK8Ou0IGAKA7Cu+PGwAAdBMUMgAABqCQAQAwAIUMAIABKGQAAAxAIQMAYAAKGQAAA1DI\nAAAY4P8DUj97e37zz4oAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7fe3a2ff4668>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "6667pZttboaj",
        "colab_type": "code",
        "outputId": "d3bdf331-e90f-448f-899e-4d0dcd252878",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "cell_type": "code",
      "source": [
        "# Bias in embeddings\n",
        "glove.most_similar(positive=['woman', 'doctor'], negative=['man'], topn=5)"
      ],
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[('nurse', 0.7735227346420288),\n",
              " ('physician', 0.7189429998397827),\n",
              " ('doctors', 0.6824328303337097),\n",
              " ('patient', 0.6750682592391968),\n",
              " ('dentist', 0.6726033687591553)]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "metadata": {
        "id": "LGOkD_DGIq7t",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Using Embeddings"
      ]
    },
    {
      "metadata": {
        "id": "tu3Bl-_yw9LA",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "There are several different ways to use embeddings. \n",
        "\n",
        "1. Use your own trained embeddings (trained on an unsupervised dataset).\n",
        "2. Use pretrained embeddings (GloVe, word2vec, etc.)\n",
        "3. Randomly initialized embeddings.\n",
        "\n",
        "Once you have chosen embeddings, you can choose to freeze them or continue to train them using the supervised data (this could lead to overfitting). In this example, we're going to use GloVe embeddings and freeze them during training. Our task will be to predict an article's category given its title."
      ]
    },
    {
      "metadata": {
        "id": "8OlZB0vc1Hbb",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Set up"
      ]
    },
    {
      "metadata": {
        "id": "11CPv1Z-IrD6",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "355684a5-6294-4f06-a475-6d124c21868f"
      },
      "cell_type": "code",
      "source": [
        "# Load PyTorch library\n",
        "!pip3 install torch"
      ],
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: torch in /usr/local/lib/python3.6/dist-packages (1.0.0)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "T3daO-e9wpXe",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import os\n",
        "from argparse import Namespace\n",
        "import collections\n",
        "import json\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import re\n",
        "import torch"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Ye64bXPrwpaQ",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Set Numpy and PyTorch seeds\n",
        "def set_seeds(seed, cuda):\n",
        "    np.random.seed(seed)\n",
        "    torch.manual_seed(seed)\n",
        "    if cuda:\n",
        "        torch.cuda.manual_seed_all(seed)\n",
        "        \n",
        "# Creating directories\n",
        "def create_dirs(dirpath):\n",
        "    if not os.path.exists(dirpath):\n",
        "        os.makedirs(dirpath)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "m5_jiImZ1NAr",
        "colab_type": "code",
        "outputId": "238e95dd-3df4-45c6-e6dd-ebdeb33b8a88",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Arguments\n",
        "args = Namespace(\n",
        "    seed=1234,\n",
        "    cuda=True,\n",
        "    shuffle=True,\n",
        "    data_file=\"news.csv\",\n",
        "    split_data_file=\"split_news.csv\",\n",
        "    vectorizer_file=\"vectorizer.json\",\n",
        "    model_state_file=\"model.pth\",\n",
        "    save_dir=\"news\",\n",
        "    train_size=0.7,\n",
        "    val_size=0.15,\n",
        "    test_size=0.15,\n",
        "    cutoff=25, # token must appear at least <cutoff> times to be in SequenceVocabulary\n",
        "    num_epochs=5,\n",
        "    early_stopping_criteria=5,\n",
        "    learning_rate=1e-3,\n",
        "    batch_size=64,\n",
        "    num_filters=100,\n",
        "    embedding_dim=100,\n",
        "    hidden_dim=100,\n",
        "    dropout_p=0.1,\n",
        ")\n",
        "\n",
        "# Set seeds\n",
        "set_seeds(seed=args.seed, cuda=args.cuda)\n",
        "\n",
        "# Create save dir\n",
        "create_dirs(args.save_dir)\n",
        "\n",
        "# Expand filepaths\n",
        "args.vectorizer_file = os.path.join(args.save_dir, args.vectorizer_file)\n",
        "args.model_state_file = os.path.join(args.save_dir, args.model_state_file)\n",
        "\n",
        "# Check CUDA\n",
        "if not torch.cuda.is_available():\n",
        "    args.cuda = False\n",
        "args.device = torch.device(\"cuda\" if args.cuda else \"cpu\")\n",
        "print(\"Using CUDA: {}\".format(args.cuda))"
      ],
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using CUDA: True\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "GkNclM3I1XMg",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Data"
      ]
    },
    {
      "metadata": {
        "id": "vk6ucvgp1NDD",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import re\n",
        "import urllib"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "uVQNGEYa1NFu",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Upload data from GitHub to notebook's local drive\n",
        "url = \"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/data/news.csv\"\n",
        "response = urllib.request.urlopen(url)\n",
        "html = response.read()\n",
        "with open(args.data_file, 'wb') as fp:\n",
        "    fp.write(html)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "aTU6ENCu1NIc",
        "colab_type": "code",
        "outputId": "6a7f2c48-4c8f-4283-accb-9d96a61ecbea",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "cell_type": "code",
      "source": [
        "# Raw data\n",
        "df = pd.read_csv(args.data_file, header=0)\n",
        "df.head()"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>category</th>\n",
              "      <th>title</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Business</td>\n",
              "      <td>Wall St. Bears Claw Back Into the Black (Reuters)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Business</td>\n",
              "      <td>Carlyle Looks Toward Commercial Aerospace (Reu...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Business</td>\n",
              "      <td>Oil and Economy Cloud Stocks' Outlook (Reuters)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Business</td>\n",
              "      <td>Iraq Halts Oil Exports from Main Southern Pipe...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Business</td>\n",
              "      <td>Oil prices soar to all-time record, posing new...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   category                                              title\n",
              "0  Business  Wall St. Bears Claw Back Into the Black (Reuters)\n",
              "1  Business  Carlyle Looks Toward Commercial Aerospace (Reu...\n",
              "2  Business    Oil and Economy Cloud Stocks' Outlook (Reuters)\n",
              "3  Business  Iraq Halts Oil Exports from Main Southern Pipe...\n",
              "4  Business  Oil prices soar to all-time record, posing new..."
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 33
        }
      ]
    },
    {
      "metadata": {
        "id": "xIouG0f71NK4",
        "colab_type": "code",
        "outputId": "1f49fb96-8ef6-4772-89f7-6d9a605cda21",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "# Split by category\n",
        "by_category = collections.defaultdict(list)\n",
        "for _, row in df.iterrows():\n",
        "    by_category[row.category].append(row.to_dict())\n",
        "for category in by_category:\n",
        "    print (\"{0}: {1}\".format(category, len(by_category[category])))"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Business: 30000\n",
            "Sci/Tech: 30000\n",
            "Sports: 30000\n",
            "World: 30000\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "YZyFpi791NNP",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Create split data\n",
        "final_list = []\n",
        "for _, item_list in sorted(by_category.items()):\n",
        "    if args.shuffle:\n",
        "        np.random.shuffle(item_list)\n",
        "    n = len(item_list)\n",
        "    n_train = int(args.train_size*n)\n",
        "    n_val = int(args.val_size*n)\n",
        "    n_test = int(args.test_size*n)\n",
        "\n",
        "  # Give data point a split attribute\n",
        "    for item in item_list[:n_train]:\n",
        "        item['split'] = 'train'\n",
        "    for item in item_list[n_train:n_train+n_val]:\n",
        "        item['split'] = 'val'\n",
        "    for item in item_list[n_train+n_val:]:\n",
        "        item['split'] = 'test'  \n",
        "\n",
        "    # Add to final list\n",
        "    final_list.extend(item_list)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "NLrQTVoO1NP4",
        "colab_type": "code",
        "outputId": "c0800a8b-0c18-450c-87bf-04971bc92075",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "# df with split datasets\n",
        "split_df = pd.DataFrame(final_list)\n",
        "split_df[\"split\"].value_counts()"
      ],
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "train    84000\n",
              "test     18000\n",
              "val      18000\n",
              "Name: split, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 36
        }
      ]
    },
    {
      "metadata": {
        "id": "v-HSMGwh1NUu",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Preprocessing\n",
        "def preprocess_text(text):\n",
        "    text = ' '.join(word.lower() for word in text.split(\" \"))\n",
        "    text = re.sub(r\"([.,!?])\", r\" \\1 \", text)\n",
        "    text = re.sub(r\"[^a-zA-Z.,!?]+\", r\" \", text)\n",
        "    return text\n",
        "    \n",
        "split_df.title = split_df.title.apply(preprocess_text)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "gvNVV7n11NXX",
        "colab_type": "code",
        "outputId": "bcb4a3dd-ba84-4475-dfe1-161f1f725684",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "cell_type": "code",
      "source": [
        "# Save to CSV\n",
        "split_df.to_csv(args.split_data_file, index=False)\n",
        "split_df.head()"
      ],
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>category</th>\n",
              "      <th>split</th>\n",
              "      <th>title</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Business</td>\n",
              "      <td>train</td>\n",
              "      <td>general electric posts higher rd quarter profit</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Business</td>\n",
              "      <td>train</td>\n",
              "      <td>lilly to eliminate up to us jobs</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Business</td>\n",
              "      <td>train</td>\n",
              "      <td>s amp p lowers america west outlook to negative</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Business</td>\n",
              "      <td>train</td>\n",
              "      <td>does rand walk the talk on labor policy ?</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Business</td>\n",
              "      <td>train</td>\n",
              "      <td>housekeeper advocates for changes</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   category  split                                            title\n",
              "0  Business  train  general electric posts higher rd quarter profit\n",
              "1  Business  train                 lilly to eliminate up to us jobs\n",
              "2  Business  train  s amp p lowers america west outlook to negative\n",
              "3  Business  train       does rand walk the talk on labor policy ? \n",
              "4  Business  train                housekeeper advocates for changes"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 38
        }
      ]
    },
    {
      "metadata": {
        "id": "cPDWGdlE5vye",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Vocabulary"
      ]
    },
    {
      "metadata": {
        "id": "EMR5Y3D95v6P",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class Vocabulary(object):\n",
        "    def __init__(self, token_to_idx=None):\n",
        "\n",
        "        # Token to index\n",
        "        if token_to_idx is None:\n",
        "            token_to_idx = {}\n",
        "        self.token_to_idx = token_to_idx\n",
        "\n",
        "        # Index to token\n",
        "        self.idx_to_token = {idx: token \\\n",
        "                             for token, idx in self.token_to_idx.items()}\n",
        "\n",
        "    def to_serializable(self):\n",
        "        return {'token_to_idx': self.token_to_idx}\n",
        "\n",
        "    @classmethod\n",
        "    def from_serializable(cls, contents):\n",
        "        return cls(**contents)\n",
        "\n",
        "    def add_token(self, token):\n",
        "        if token in self.token_to_idx:\n",
        "            index = self.token_to_idx[token]\n",
        "        else:\n",
        "            index = len(self.token_to_idx)\n",
        "            self.token_to_idx[token] = index\n",
        "            self.idx_to_token[index] = token\n",
        "        return index\n",
        "\n",
        "    def add_tokens(self, tokens):\n",
        "        return [self.add_token[token] for token in tokens]\n",
        "\n",
        "    def lookup_token(self, token):\n",
        "        return self.token_to_idx[token]\n",
        "\n",
        "    def lookup_index(self, index):\n",
        "        if index not in self.idx_to_token:\n",
        "            raise KeyError(\"the index (%d) is not in the Vocabulary\" % index)\n",
        "        return self.idx_to_token[index]\n",
        "\n",
        "    def __str__(self):\n",
        "        return \"<Vocabulary(size=%d)>\" % len(self)\n",
        "\n",
        "    def __len__(self):\n",
        "        return len(self.token_to_idx)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "WKnfFKY95v81",
        "colab_type": "code",
        "outputId": "03e03491-84aa-45f8-e004-1e9ba58c19c5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "# Vocabulary instance\n",
        "category_vocab = Vocabulary()\n",
        "for index, row in df.iterrows():\n",
        "    category_vocab.add_token(row.category)\n",
        "print (category_vocab) # __str__\n",
        "print (len(category_vocab)) # __len__\n",
        "index = category_vocab.lookup_token(\"Business\")\n",
        "print (index)\n",
        "print (category_vocab.lookup_index(index))"
      ],
      "execution_count": 60,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<Vocabulary(size=4)>\n",
            "4\n",
            "0\n",
            "Business\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "Th-KJnOi_74q",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Sequence vocabulary"
      ]
    },
    {
      "metadata": {
        "id": "jxO9-6vhBCSO",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Next, we're going to create our Vocabulary classes for the article's title, which is a sequence of tokens."
      ]
    },
    {
      "metadata": {
        "id": "ip8ViI2v_7Y2",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from collections import Counter\n",
        "import string"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "S2LTR5NjBDCG",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class SequenceVocabulary(Vocabulary):\n",
        "    def __init__(self, token_to_idx=None, unk_token=\"<UNK>\",\n",
        "                 mask_token=\"<MASK>\", begin_seq_token=\"<BEGIN>\",\n",
        "                 end_seq_token=\"<END>\"):\n",
        "\n",
        "        super(SequenceVocabulary, self).__init__(token_to_idx)\n",
        "\n",
        "        self.mask_token = mask_token\n",
        "        self.unk_token = unk_token\n",
        "        self.begin_seq_token = begin_seq_token\n",
        "        self.end_seq_token = end_seq_token\n",
        "\n",
        "        self.mask_index = self.add_token(self.mask_token)\n",
        "        self.unk_index = self.add_token(self.unk_token)\n",
        "        self.begin_seq_index = self.add_token(self.begin_seq_token)\n",
        "        self.end_seq_index = self.add_token(self.end_seq_token)\n",
        "        \n",
        "        # Index to token\n",
        "        self.idx_to_token = {idx: token \\\n",
        "                             for token, idx in self.token_to_idx.items()}\n",
        "\n",
        "    def to_serializable(self):\n",
        "        contents = super(SequenceVocabulary, self).to_serializable()\n",
        "        contents.update({'unk_token': self.unk_token,\n",
        "                         'mask_token': self.mask_token,\n",
        "                         'begin_seq_token': self.begin_seq_token,\n",
        "                         'end_seq_token': self.end_seq_token})\n",
        "        return contents\n",
        "\n",
        "    def lookup_token(self, token):\n",
        "        return self.token_to_idx.get(token, self.unk_index)\n",
        "    \n",
        "    def lookup_index(self, index):\n",
        "        if index not in self.idx_to_token:\n",
        "            raise KeyError(\"the index (%d) is not in the SequenceVocabulary\" % index)\n",
        "        return self.idx_to_token[index]\n",
        "    \n",
        "    def __str__(self):\n",
        "        return \"<SequenceVocabulary(size=%d)>\" % len(self.token_to_idx)\n",
        "\n",
        "    def __len__(self):\n",
        "        return len(self.token_to_idx)\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "DvDPVDbsBDE4",
        "colab_type": "code",
        "outputId": "09913e32-1588-4410-9d22-6f4024ac3193",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "# Get word counts\n",
        "word_counts = Counter()\n",
        "for title in split_df.title:\n",
        "    for token in title.split(\" \"):\n",
        "        if token not in string.punctuation:\n",
        "            word_counts[token] += 1\n",
        "\n",
        "# Create SequenceVocabulary instance\n",
        "title_vocab = SequenceVocabulary()\n",
        "for word, word_count in word_counts.items():\n",
        "    if word_count >= args.cutoff:\n",
        "        title_vocab.add_token(word)\n",
        "print (title_vocab) # __str__\n",
        "print (len(title_vocab)) # __len__\n",
        "index = title_vocab.lookup_token(\"general\")\n",
        "print (index)\n",
        "print (title_vocab.lookup_index(index))"
      ],
      "execution_count": 63,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<SequenceVocabulary(size=4400)>\n",
            "4400\n",
            "4\n",
            "measures\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "6TuUHvlI6JGh",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Vectorizer"
      ]
    },
    {
      "metadata": {
        "id": "s8IsOPHT5v_i",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class NewsVectorizer(object):\n",
        "    def __init__(self, title_vocab, category_vocab):\n",
        "        self.title_vocab = title_vocab\n",
        "        self.category_vocab = category_vocab\n",
        "\n",
        "    def vectorize(self, title):\n",
        "        indices = [self.title_vocab.lookup_token(token) for token in title.split(\" \")]\n",
        "        indices = [self.title_vocab.begin_seq_index] + indices + \\\n",
        "            [self.title_vocab.end_seq_index]\n",
        "        \n",
        "        # Create vector\n",
        "        title_length = len(indices)\n",
        "        vector = np.zeros(title_length, dtype=np.int64)\n",
        "        vector[:len(indices)] = indices\n",
        "\n",
        "        return vector\n",
        "    \n",
        "    def unvectorize(self, vector):\n",
        "        tokens = [self.title_vocab.lookup_index(index) for index in vector]\n",
        "        title = \" \".join(token for token in tokens)\n",
        "        return title\n",
        "\n",
        "    @classmethod\n",
        "    def from_dataframe(cls, df, cutoff):\n",
        "        \n",
        "        # Create class vocab\n",
        "        category_vocab = Vocabulary()        \n",
        "        for category in sorted(set(df.category)):\n",
        "            category_vocab.add_token(category)\n",
        "\n",
        "        # Get word counts\n",
        "        word_counts = Counter()\n",
        "        for title in df.title:\n",
        "            for token in title.split(\" \"):\n",
        "                word_counts[token] += 1\n",
        "        \n",
        "        # Create title vocab\n",
        "        title_vocab = SequenceVocabulary()\n",
        "        for word, word_count in word_counts.items():\n",
        "            if word_count >= cutoff:\n",
        "                title_vocab.add_token(word)\n",
        "        \n",
        "        return cls(title_vocab, category_vocab)\n",
        "\n",
        "    @classmethod\n",
        "    def from_serializable(cls, contents):\n",
        "        title_vocab = SequenceVocabulary.from_serializable(contents['title_vocab'])\n",
        "        category_vocab = Vocabulary.from_serializable(contents['category_vocab'])\n",
        "        return cls(title_vocab=title_vocab, category_vocab=category_vocab)\n",
        "    \n",
        "    def to_serializable(self):\n",
        "        return {'title_vocab': self.title_vocab.to_serializable(),\n",
        "                'category_vocab': self.category_vocab.to_serializable()}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "p2blo9sT5wCS",
        "colab_type": "code",
        "outputId": "8a134781-caca-47b7-e7cc-913ca3a1d9fa",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "cell_type": "code",
      "source": [
        "# Vectorizer instance\n",
        "vectorizer = NewsVectorizer.from_dataframe(split_df, cutoff=args.cutoff)\n",
        "print (vectorizer.title_vocab)\n",
        "print (vectorizer.category_vocab)\n",
        "vectorized_title = vectorizer.vectorize(preprocess_text(\n",
        "    \"Roger Federer wins the Wimbledon tennis tournament.\"))\n",
        "print (np.shape(vectorized_title))\n",
        "print (vectorized_title)\n",
        "print (vectorizer.unvectorize(vectorized_title))"
      ],
      "execution_count": 65,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<SequenceVocabulary(size=4405)>\n",
            "<Vocabulary(size=4)>\n",
            "(11,)\n",
            "[   2    1 4152 1232   25    1 2393 4077   39   31    3]\n",
            "<BEGIN> <UNK> federer wins the <UNK> tennis tournament .  <END>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "jTpYV1FkHBTi",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Dataset"
      ]
    },
    {
      "metadata": {
        "id": "_PcKZGbYIVA7",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from torch.utils.data import Dataset, DataLoader"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Lrssjuxb1NZy",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class NewsDataset(Dataset):\n",
        "    def __init__(self, df, vectorizer):\n",
        "        self.df = df\n",
        "        self.vectorizer = vectorizer\n",
        "        \n",
        "        # Max title length\n",
        "        get_length = lambda title: len(title.split(\" \"))\n",
        "        self.max_seq_length = max(map(get_length, df.title)) + 2 # (<BEGIN> + <END>)\n",
        "\n",
        "        # Data splits\n",
        "        self.train_df = self.df[self.df.split=='train']\n",
        "        self.train_size = len(self.train_df)\n",
        "        self.val_df = self.df[self.df.split=='val']\n",
        "        self.val_size = len(self.val_df)\n",
        "        self.test_df = self.df[self.df.split=='test']\n",
        "        self.test_size = len(self.test_df)\n",
        "        self.lookup_dict = {'train': (self.train_df, self.train_size), \n",
        "                            'val': (self.val_df, self.val_size),\n",
        "                            'test': (self.test_df, self.test_size)}\n",
        "        self.set_split('train')\n",
        "\n",
        "        # Class weights (for imbalances)\n",
        "        class_counts = df.category.value_counts().to_dict()\n",
        "        def sort_key(item):\n",
        "            return self.vectorizer.category_vocab.lookup_token(item[0])\n",
        "        sorted_counts = sorted(class_counts.items(), key=sort_key)\n",
        "        frequencies = [count for _, count in sorted_counts]\n",
        "        self.class_weights = 1.0 / torch.tensor(frequencies, dtype=torch.float32)\n",
        "\n",
        "    @classmethod\n",
        "    def load_dataset_and_make_vectorizer(cls, split_data_file, cutoff):\n",
        "        df = pd.read_csv(split_data_file, header=0)\n",
        "        train_df = df[df.split=='train']\n",
        "        return cls(df, NewsVectorizer.from_dataframe(train_df, cutoff))\n",
        "\n",
        "    @classmethod\n",
        "    def load_dataset_and_load_vectorizer(cls, split_data_file, vectorizer_filepath):\n",
        "        df = pd.read_csv(split_data_file, header=0)\n",
        "        vectorizer = cls.load_vectorizer_only(vectorizer_filepath)\n",
        "        return cls(df, vectorizer)\n",
        "\n",
        "    def load_vectorizer_only(vectorizer_filepath):\n",
        "        with open(vectorizer_filepath) as fp:\n",
        "            return NewsVectorizer.from_serializable(json.load(fp))\n",
        "\n",
        "    def save_vectorizer(self, vectorizer_filepath):\n",
        "        with open(vectorizer_filepath, \"w\") as fp:\n",
        "            json.dump(self.vectorizer.to_serializable(), fp)\n",
        "\n",
        "    def set_split(self, split=\"train\"):\n",
        "        self.target_split = split\n",
        "        self.target_df, self.target_size = self.lookup_dict[split]\n",
        "\n",
        "    def __str__(self):\n",
        "        return \"<Dataset(split={0}, size={1})\".format(\n",
        "            self.target_split, self.target_size)\n",
        "\n",
        "    def __len__(self):\n",
        "        return self.target_size\n",
        "\n",
        "    def __getitem__(self, index):\n",
        "        row = self.target_df.iloc[index]\n",
        "        title_vector = self.vectorizer.vectorize(row.title)\n",
        "        category_index = self.vectorizer.category_vocab.lookup_token(row.category)\n",
        "        return {'title': title_vector, 'category': category_index}\n",
        "\n",
        "    def get_num_batches(self, batch_size):\n",
        "        return len(self) // batch_size\n",
        "\n",
        "    def generate_batches(self, batch_size, collate_fn, shuffle=True, \n",
        "                         drop_last=False, device=\"cpu\"):\n",
        "        dataloader = DataLoader(dataset=self, batch_size=batch_size,\n",
        "                                collate_fn=collate_fn, shuffle=shuffle, \n",
        "                                drop_last=drop_last)\n",
        "        for data_dict in dataloader:\n",
        "            out_data_dict = {}\n",
        "            for name, tensor in data_dict.items():\n",
        "                out_data_dict[name] = data_dict[name].to(device)\n",
        "            yield out_data_dict"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "e9I-AFKsIP2e",
        "colab_type": "code",
        "outputId": "913612d9-805a-4c5c-df5a-578fd1b8c935",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "# Dataset instance\n",
        "dataset = NewsDataset.load_dataset_and_make_vectorizer(args.split_data_file, \n",
        "                                                       cutoff=args.cutoff)\n",
        "print (dataset) # __str__\n",
        "title_vector = dataset[5]['title'] # __getitem__\n",
        "print (title_vector)\n",
        "print (dataset.vectorizer.unvectorize(title_vector))\n",
        "print (dataset.class_weights)"
      ],
      "execution_count": 68,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<Dataset(split=train, size=84000)\n",
            "[ 2 32 33 10 34 13  3]\n",
            "<BEGIN> software firm to cut jobs <END>\n",
            "tensor([3.3333e-05, 3.3333e-05, 3.3333e-05, 3.3333e-05])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "IxjCMr_DINGk",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Model"
      ]
    },
    {
      "metadata": {
        "id": "202KnN2lLj61",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "input → embedding → conv → FC \n",
        "\n",
        "We will be using 1d conv operations ([nn.Conv1D](https://pytorch.org/docs/stable/nn.html#torch.nn.Conv1d)) even though our inputs are words because we are not representing them at a character level. The inputs are of shape $\\in \\mathbb{R}^{NXSXE}$\n",
        "* where:\n",
        "    * N = batchsize\n",
        "    * S = max sentence length \n",
        "    * E = embedding dim at a word level"
      ]
    },
    {
      "metadata": {
        "id": "j-Xgp0F3INRA",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import torch.nn as nn\n",
        "import torch.nn.functional as F"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Lfw7csveHApA",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class NewsModel(nn.Module):\n",
        "    def __init__(self, embedding_dim, num_embeddings, num_input_channels, \n",
        "                 num_channels, hidden_dim, num_classes, dropout_p, \n",
        "                 pretrained_embeddings=None, freeze_embeddings=False,\n",
        "                 padding_idx=0):\n",
        "        super(NewsModel, self).__init__()\n",
        "        \n",
        "        if pretrained_embeddings is None:\n",
        "            self.embeddings = nn.Embedding(embedding_dim=embedding_dim,\n",
        "                                          num_embeddings=num_embeddings,\n",
        "                                          padding_idx=padding_idx)\n",
        "        else:\n",
        "            pretrained_embeddings = torch.from_numpy(pretrained_embeddings).float()\n",
        "            self.embeddings = nn.Embedding(embedding_dim=embedding_dim,\n",
        "                                           num_embeddings=num_embeddings,\n",
        "                                           padding_idx=padding_idx,\n",
        "                                           _weight=pretrained_embeddings)\n",
        "        \n",
        "        # Conv weights\n",
        "        self.conv = nn.ModuleList([nn.Conv1d(num_input_channels, num_channels, \n",
        "                                             kernel_size=f) for f in [2,3,4]])\n",
        "     \n",
        "        # FC weights\n",
        "        self.dropout = nn.Dropout(dropout_p)\n",
        "        self.fc1 = nn.Linear(num_channels*3, hidden_dim)\n",
        "        self.fc2 = nn.Linear(hidden_dim, num_classes)\n",
        "        \n",
        "        if freeze_embeddings:\n",
        "            self.embeddings.weight.requires_grad = False\n",
        "\n",
        "    def forward(self, x_in, channel_first=False, apply_softmax=False):\n",
        "        \n",
        "        # Embed\n",
        "        x_in = self.embeddings(x_in)\n",
        "\n",
        "        # Rearrange input so num_channels is in dim 1 (N, C, L)\n",
        "        if not channel_first:\n",
        "            x_in = x_in.transpose(1, 2)\n",
        "            \n",
        "        # Conv outputs\n",
        "        z1 = self.conv[0](x_in)\n",
        "        z1 = F.max_pool1d(z1, z1.size(2)).squeeze(2)\n",
        "        z2 = self.conv[1](x_in)\n",
        "        z2 = F.max_pool1d(z2, z2.size(2)).squeeze(2)\n",
        "        z3 = self.conv[2](x_in)\n",
        "        z3 = F.max_pool1d(z3, z3.size(2)).squeeze(2)\n",
        "        \n",
        "        # Concat conv outputs\n",
        "        z = torch.cat([z1, z2, z3], 1)\n",
        "\n",
        "        # FC layers\n",
        "        z = self.dropout(z)\n",
        "        z = self.fc1(z)\n",
        "        y_pred = self.fc2(z)\n",
        "        \n",
        "        if apply_softmax:\n",
        "            y_pred = F.softmax(y_pred, dim=1)\n",
        "        return y_pred"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "d7_4SiLkKji8",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Training"
      ]
    },
    {
      "metadata": {
        "id": "UQAIfKN2HAtN",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import torch.optim as optim"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "RwgZASWYHAwB",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class Trainer(object):\n",
        "    def __init__(self, dataset, model, model_state_file, save_dir, device, shuffle, \n",
        "               num_epochs, batch_size, learning_rate, early_stopping_criteria):\n",
        "        self.dataset = dataset\n",
        "        self.class_weights = dataset.class_weights.to(device)\n",
        "        self.model = model.to(device)\n",
        "        self.save_dir = save_dir\n",
        "        self.device = device\n",
        "        self.shuffle = shuffle\n",
        "        self.num_epochs = num_epochs\n",
        "        self.batch_size = batch_size\n",
        "        self.loss_func = nn.CrossEntropyLoss(self.class_weights)\n",
        "        self.optimizer = optim.Adam(self.model.parameters(), lr=learning_rate)\n",
        "        self.scheduler = optim.lr_scheduler.ReduceLROnPlateau(\n",
        "            optimizer=self.optimizer, mode='min', factor=0.5, patience=1)\n",
        "        self.train_state = {\n",
        "            'stop_early': False, \n",
        "            'early_stopping_step': 0,\n",
        "            'early_stopping_best_val': 1e8,\n",
        "            'early_stopping_criteria': early_stopping_criteria,\n",
        "            'learning_rate': learning_rate,\n",
        "            'epoch_index': 0,\n",
        "            'train_loss': [],\n",
        "            'train_acc': [],\n",
        "            'val_loss': [],\n",
        "            'val_acc': [],\n",
        "            'test_loss': -1,\n",
        "            'test_acc': -1,\n",
        "            'model_filename': model_state_file}\n",
        "    \n",
        "    def update_train_state(self):\n",
        "\n",
        "        # Verbose\n",
        "        print (\"[EPOCH]: {0:02d} | [LR]: {1} | [TRAIN LOSS]: {2:.2f} | [TRAIN ACC]: {3:.1f}% | [VAL LOSS]: {4:.2f} | [VAL ACC]: {5:.1f}%\".format(\n",
        "          self.train_state['epoch_index'], self.train_state['learning_rate'], \n",
        "            self.train_state['train_loss'][-1], self.train_state['train_acc'][-1], \n",
        "            self.train_state['val_loss'][-1], self.train_state['val_acc'][-1]))\n",
        "\n",
        "        # Save one model at least\n",
        "        if self.train_state['epoch_index'] == 0:\n",
        "            torch.save(self.model.state_dict(), self.train_state['model_filename'])\n",
        "            self.train_state['stop_early'] = False\n",
        "\n",
        "        # Save model if performance improved\n",
        "        elif self.train_state['epoch_index'] >= 1:\n",
        "            loss_tm1, loss_t = self.train_state['val_loss'][-2:]\n",
        "\n",
        "            # If loss worsened\n",
        "            if loss_t >= self.train_state['early_stopping_best_val']:\n",
        "                # Update step\n",
        "                self.train_state['early_stopping_step'] += 1\n",
        "\n",
        "            # Loss decreased\n",
        "            else:\n",
        "                # Save the best model\n",
        "                if loss_t < self.train_state['early_stopping_best_val']:\n",
        "                    torch.save(self.model.state_dict(), self.train_state['model_filename'])\n",
        "\n",
        "                # Reset early stopping step\n",
        "                self.train_state['early_stopping_step'] = 0\n",
        "\n",
        "            # Stop early ?\n",
        "            self.train_state['stop_early'] = self.train_state['early_stopping_step'] \\\n",
        "              >= self.train_state['early_stopping_criteria']\n",
        "        return self.train_state\n",
        "  \n",
        "    def compute_accuracy(self, y_pred, y_target):\n",
        "        _, y_pred_indices = y_pred.max(dim=1)\n",
        "        n_correct = torch.eq(y_pred_indices, y_target).sum().item()\n",
        "        return n_correct / len(y_pred_indices) * 100\n",
        "    \n",
        "    def pad_seq(self, seq, length):\n",
        "        vector = np.zeros(length, dtype=np.int64)\n",
        "        vector[:len(seq)] = seq\n",
        "        vector[len(seq):] = self.dataset.vectorizer.title_vocab.mask_index\n",
        "        return vector\n",
        "    \n",
        "    def collate_fn(self, batch):\n",
        "        \n",
        "        # Make a deep copy\n",
        "        batch_copy = copy.deepcopy(batch)\n",
        "        processed_batch = {\"title\": [], \"category\": []}\n",
        "        \n",
        "        # Get max sequence length\n",
        "        max_seq_len = max([len(sample[\"title\"]) for sample in batch_copy])\n",
        "        \n",
        "        # Pad\n",
        "        for i, sample in enumerate(batch_copy):\n",
        "            seq = sample[\"title\"]\n",
        "            category = sample[\"category\"]\n",
        "            padded_seq = self.pad_seq(seq, max_seq_len)\n",
        "            processed_batch[\"title\"].append(padded_seq)\n",
        "            processed_batch[\"category\"].append(category)\n",
        "            \n",
        "        # Convert to appropriate tensor types\n",
        "        processed_batch[\"title\"] = torch.LongTensor(\n",
        "            processed_batch[\"title\"])\n",
        "        processed_batch[\"category\"] = torch.LongTensor(\n",
        "            processed_batch[\"category\"])\n",
        "        \n",
        "        return processed_batch    \n",
        "  \n",
        "    def run_train_loop(self):\n",
        "        for epoch_index in range(self.num_epochs):\n",
        "            self.train_state['epoch_index'] = epoch_index\n",
        "      \n",
        "            # Iterate over train dataset\n",
        "\n",
        "            # initialize batch generator, set loss and acc to 0, set train mode on\n",
        "            self.dataset.set_split('train')\n",
        "            batch_generator = self.dataset.generate_batches(\n",
        "                batch_size=self.batch_size, collate_fn=self.collate_fn, \n",
        "                shuffle=self.shuffle, device=self.device)\n",
        "            running_loss = 0.0\n",
        "            running_acc = 0.0\n",
        "            self.model.train()\n",
        "\n",
        "            for batch_index, batch_dict in enumerate(batch_generator):\n",
        "                # zero the gradients\n",
        "                self.optimizer.zero_grad()\n",
        "\n",
        "                # compute the output\n",
        "                y_pred = self.model(batch_dict['title'])\n",
        "\n",
        "                # compute the loss\n",
        "                loss = self.loss_func(y_pred, batch_dict['category'])\n",
        "                loss_t = loss.item()\n",
        "                running_loss += (loss_t - running_loss) / (batch_index + 1)\n",
        "\n",
        "                # compute gradients using loss\n",
        "                loss.backward()\n",
        "\n",
        "                # use optimizer to take a gradient step\n",
        "                self.optimizer.step()\n",
        "                \n",
        "                # compute the accuracy\n",
        "                acc_t = self.compute_accuracy(y_pred, batch_dict['category'])\n",
        "                running_acc += (acc_t - running_acc) / (batch_index + 1)\n",
        "\n",
        "            self.train_state['train_loss'].append(running_loss)\n",
        "            self.train_state['train_acc'].append(running_acc)\n",
        "\n",
        "            # Iterate over val dataset\n",
        "\n",
        "            # initialize batch generator, set loss and acc to 0; set eval mode on\n",
        "            self.dataset.set_split('val')\n",
        "            batch_generator = self.dataset.generate_batches(\n",
        "                batch_size=self.batch_size, collate_fn=self.collate_fn, \n",
        "                shuffle=self.shuffle, device=self.device)\n",
        "            running_loss = 0.\n",
        "            running_acc = 0.\n",
        "            self.model.eval()\n",
        "\n",
        "            for batch_index, batch_dict in enumerate(batch_generator):\n",
        "\n",
        "                # compute the output\n",
        "                y_pred =  self.model(batch_dict['title'])\n",
        "\n",
        "                # compute the loss\n",
        "                loss = self.loss_func(y_pred, batch_dict['category'])\n",
        "                loss_t = loss.to(\"cpu\").item()\n",
        "                running_loss += (loss_t - running_loss) / (batch_index + 1)\n",
        "\n",
        "                # compute the accuracy\n",
        "                acc_t = self.compute_accuracy(y_pred, batch_dict['category'])\n",
        "                running_acc += (acc_t - running_acc) / (batch_index + 1)\n",
        "\n",
        "            self.train_state['val_loss'].append(running_loss)\n",
        "            self.train_state['val_acc'].append(running_acc)\n",
        "\n",
        "            self.train_state = self.update_train_state()\n",
        "            self.scheduler.step(self.train_state['val_loss'][-1])\n",
        "            if self.train_state['stop_early']:\n",
        "                break\n",
        "          \n",
        "    def run_test_loop(self):\n",
        "        # initialize batch generator, set loss and acc to 0; set eval mode on\n",
        "        self.dataset.set_split('test')\n",
        "        batch_generator = self.dataset.generate_batches(\n",
        "                batch_size=self.batch_size, collate_fn=self.collate_fn, \n",
        "                shuffle=self.shuffle, device=self.device)\n",
        "        running_loss = 0.0\n",
        "        running_acc = 0.0\n",
        "        self.model.eval()\n",
        "\n",
        "        for batch_index, batch_dict in enumerate(batch_generator):\n",
        "            # compute the output\n",
        "            y_pred =  self.model(batch_dict['title'])\n",
        "\n",
        "            # compute the loss\n",
        "            loss = self.loss_func(y_pred, batch_dict['category'])\n",
        "            loss_t = loss.item()\n",
        "            running_loss += (loss_t - running_loss) / (batch_index + 1)\n",
        "\n",
        "            # compute the accuracy\n",
        "            acc_t = self.compute_accuracy(y_pred, batch_dict['category'])\n",
        "            running_acc += (acc_t - running_acc) / (batch_index + 1)\n",
        "\n",
        "        self.train_state['test_loss'] = running_loss\n",
        "        self.train_state['test_acc'] = running_acc\n",
        "    \n",
        "    def plot_performance(self):\n",
        "        # Figure size\n",
        "        plt.figure(figsize=(15,5))\n",
        "\n",
        "        # Plot Loss\n",
        "        plt.subplot(1, 2, 1)\n",
        "        plt.title(\"Loss\")\n",
        "        plt.plot(trainer.train_state[\"train_loss\"], label=\"train\")\n",
        "        plt.plot(trainer.train_state[\"val_loss\"], label=\"val\")\n",
        "        plt.legend(loc='upper right')\n",
        "\n",
        "        # Plot Accuracy\n",
        "        plt.subplot(1, 2, 2)\n",
        "        plt.title(\"Accuracy\")\n",
        "        plt.plot(trainer.train_state[\"train_acc\"], label=\"train\")\n",
        "        plt.plot(trainer.train_state[\"val_acc\"], label=\"val\")\n",
        "        plt.legend(loc='lower right')\n",
        "\n",
        "        # Save figure\n",
        "        plt.savefig(os.path.join(self.save_dir, \"performance.png\"))\n",
        "\n",
        "        # Show plots\n",
        "        plt.show()\n",
        "    \n",
        "    def save_train_state(self):\n",
        "        with open(os.path.join(self.save_dir, \"train_state.json\"), \"w\") as fp:\n",
        "            json.dump(self.train_state, fp)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "LJfKGc9cHA0y",
        "colab_type": "code",
        "outputId": "2185fea4-a2c6-4986-80d4-c781ab1ea394",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 221
        }
      },
      "cell_type": "code",
      "source": [
        "# Initialization\n",
        "dataset = NewsDataset.load_dataset_and_make_vectorizer(args.split_data_file, \n",
        "                                                       cutoff=args.cutoff)\n",
        "dataset.save_vectorizer(args.vectorizer_file)\n",
        "vectorizer = dataset.vectorizer\n",
        "model = NewsModel(embedding_dim=args.embedding_dim, \n",
        "                  num_embeddings=len(vectorizer.title_vocab), \n",
        "                  num_input_channels=args.embedding_dim, \n",
        "                  num_channels=args.num_filters, hidden_dim=args.hidden_dim, \n",
        "                  num_classes=len(vectorizer.category_vocab), \n",
        "                  dropout_p=args.dropout_p, pretrained_embeddings=None, \n",
        "                  padding_idx=vectorizer.title_vocab.mask_index)\n",
        "print (model.named_modules)"
      ],
      "execution_count": 73,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Creating from scratch!\n",
            "<bound method Module.named_modules of NewsModel(\n",
            "  (embeddings): Embedding(3407, 100, padding_idx=0)\n",
            "  (conv): ModuleList(\n",
            "    (0): Conv1d(100, 100, kernel_size=(2,), stride=(1,))\n",
            "    (1): Conv1d(100, 100, kernel_size=(3,), stride=(1,))\n",
            "    (2): Conv1d(100, 100, kernel_size=(4,), stride=(1,))\n",
            "  )\n",
            "  (dropout): Dropout(p=0.1)\n",
            "  (fc1): Linear(in_features=300, out_features=100, bias=True)\n",
            "  (fc2): Linear(in_features=100, out_features=4, bias=True)\n",
            ")>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "wDxPyGIVKoUK",
        "colab_type": "code",
        "outputId": "fabb05e9-25b3-43ea-ce72-713a0bae1426",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "cell_type": "code",
      "source": [
        "# Train\n",
        "trainer = Trainer(dataset=dataset, model=model, \n",
        "                  model_state_file=args.model_state_file, \n",
        "                  save_dir=args.save_dir, device=args.device,\n",
        "                  shuffle=args.shuffle, num_epochs=args.num_epochs, \n",
        "                  batch_size=args.batch_size, learning_rate=args.learning_rate, \n",
        "                  early_stopping_criteria=args.early_stopping_criteria)\n",
        "trainer.run_train_loop()"
      ],
      "execution_count": 58,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[EPOCH]: 00 | [LR]: 0.001 | [TRAIN LOSS]: 0.77 | [TRAIN ACC]: 69.6% | [VAL LOSS]: 0.55 | [VAL ACC]: 79.5%\n",
            "[EPOCH]: 01 | [LR]: 0.001 | [TRAIN LOSS]: 0.50 | [TRAIN ACC]: 82.0% | [VAL LOSS]: 0.51 | [VAL ACC]: 81.5%\n",
            "[EPOCH]: 02 | [LR]: 0.001 | [TRAIN LOSS]: 0.42 | [TRAIN ACC]: 84.9% | [VAL LOSS]: 0.49 | [VAL ACC]: 82.4%\n",
            "[EPOCH]: 03 | [LR]: 0.001 | [TRAIN LOSS]: 0.36 | [TRAIN ACC]: 86.9% | [VAL LOSS]: 0.49 | [VAL ACC]: 82.7%\n",
            "[EPOCH]: 04 | [LR]: 0.001 | [TRAIN LOSS]: 0.31 | [TRAIN ACC]: 88.5% | [VAL LOSS]: 0.51 | [VAL ACC]: 82.7%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "V9Ruxpc8WT17",
        "colab_type": "code",
        "outputId": "f776e4cb-0d62-4716-b51a-f5039e34d884",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 335
        }
      },
      "cell_type": "code",
      "source": [
        "# Plot performance\n",
        "trainer.plot_performance()"
      ],
      "execution_count": 74,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2gAAAE+CAYAAAD4XjP+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xlc1Peh9v3PDMO+DjCsIjuiqLjg\nbtwiatTsMatp2rR3n+ak5zz3OWkec+hpTXLSpr3v2tM0p02XkzZdsrUpas1Wl6jRaOKOirK6ISAw\nrCI7zPPHkFHcF2AYuN6vV15hfjO/mWuIEa75fReDzWazISIiIiIiIk5ndHYAERERERERsVNBExER\nERERGSBU0ERERERERAYIFTQREREREZEBQgVNRERERERkgFBBExERERERGSBU0ERu0ogRIzhz5oyz\nY4iIiPSLhx9+mLvuusvZMUQGPRU0EREREbmqgoIC/P39iYqKYv/+/c6OIzKoqaCJ9LLW1la+//3v\ns3DhQu644w5+9KMf0dnZCcCf//xn7rjjDhYtWsQDDzxAYWHhVY+LiIgMBKtXr2bRokUsXbqUNWvW\nOI6vWbOGhQsXsnDhQp599lna2tquePyLL74gMzPTce6Ft1999VX+4z/+gwceeIA33niDrq4uXnjh\nBRYuXMi8efN49tlnaW9vB6CmpoZvfetb3H777dx5551s376dLVu2sHTp0h6Z77vvPjZu3NjX3xqR\nXmdydgCRweYPf/gDZ86c4YMPPqCjo4Ply5fz/vvvc/vtt/PKK6+wefNm/Pz8+Oijj9iyZQuRkZGX\nPZ6cnOzstyIiIkJnZycbNmzg6aefxs3NjVWrVtHW1kZlZSU//vGPWbNmDWFhYfzzP/8zf/zjH1m0\naNFlj48ZM+aqr7N161bWrl1LcHAw//jHP9izZw/vv/8+XV1d3HvvvXz44YfcfffdrFq1isTERH71\nq19x5MgRvva1r7Ft2zaqqqrIy8sjNTWVsrIyTp06xaxZs/rpuyTSe1TQRHrZli1bePLJJzGZTJhM\nJu68804+++wzFi9ejMFg4L333mPp0qXccccdALS3t1/2uIiIyECwfft2xowZg5+fHwCTJ09m8+bN\n1NXVMX78eMLDwwFYtWoVbm5u/O1vf7vs8b179171ddLT0wkODgZg4cKFzJ07F3d3dwDGjBlDSUkJ\nYC9yv/3tbwEYNWoUmzZtwsPDg4ULF/LBBx+QmprKxo0buf322/Hw8Oj9b4hIH9MQR5FeVlNTQ2Bg\noON2YGAg1dXVuLu788Ybb7Bv3z4WLlzIo48+Sn5+/hWPi4iIDATZ2dls2bKFjIwMMjIyWL9+PatX\nr6a2tpaAgADH4zw9PTGZTFc8fi0X/uysqalhxYoVLFy4kEWLFrFp0yZsNhsAdXV1+Pv7Ox77ZXFc\nsmQJH3zwAQAbN25k8eLFt/bGRZxEBU2kl4WGhlJXV+e4XVdXR2hoKGD/pO/nP/85O3fuZObMmaxc\nufKqx0VERJypvr6eXbt28cUXX7Bnzx727NnD7t27OXToEEajkdraWsdjGxsbsVqtmM3myx53c3Nz\nzMkGaGhouOLr/td//Rcmk4l169bx8ccfM3v2bMd9QUFBPZ7/9OnTtLe3M2nSJDo6Oti8eTOFhYVM\nnz69t74NIv1KBU2kl82ZM4f33nuPzs5OmpqaWLt2LbNnzyY/P59/+Zd/oa2tDQ8PD0aPHo3BYLji\ncREREWf74IMPmDp1ao+hgiaTiZkzZ9LW1sa+ffs4ffo0NpuNlStX8t577zF79uzLHrdYLFRVVVFd\nXU1nZyfr1q274utWV1eTkpKCh4cHeXl57N+/n6amJgDmzZvH6tWrASgqKuK+++6js7MTo9HI4sWL\n+c///E/mzZvnGB4p4mo0B03kFjz++OO4ubk5br/00ks8/vjjlJSUsGTJEgwGA4sWLXLMKxs2bBhL\nly7F3d0dX19fvv/975OSknLZ4yIiIs62Zs0annjiiUuOZ2Zm8stf/pIXX3yRJ554Ajc3N8aMGcPX\nvvY1PD09r3j8/vvv55577iEqKoq7776bo0ePXvZ1n3zySVasWEF2djYZGRmsWLGC7373u4wdO5Zn\nn32WFStWMG/ePHx9ffnJT36Cl5cXYB/m+Pvf/17DG8WlGWxfDugVEREREXFhVquVe++9ly1btvT4\nAFXElWiIo4iIiIgMCj//+c955JFHVM7EpamgiYiIiIhLs1qt3H777VitVp588klnxxG5JRriKCIi\nIiIiMkDoCpqIiIiIiMgAoYImIiIiIiIyQPT7MvtVVWdv+TnMZh9qa5t6IU3/cKW8yto3XCkruFZe\nZe0bvZXVYvHvhTRDx1D7GelKWcG18ipr31DWvuNKeXsj69V+PrrkFTSTybVW5nGlvMraN1wpK7hW\nXmXtG66UVXpypf92rpQVXCuvsvYNZe07rpS3r7O6ZEETEREREREZjFTQREREREREBggVNBERERER\nkQFCBU1ERERERGSAUEETEREREREZIFTQREREREREBggVNBERERERkQFCBU1EZAjZsmXTdT3uBz/4\nAWVlpX2cRkRERC6mgiYiMkSUl5exceM/ruux3/3ud4mKiu7jRCIiInIxk7MD3KjWtk7WbTtGerwZ\nb0+Xiy8i4jQ//emPOXo0l9tum8SCBXdQXl7Gz372S15++UWqqippbm7mySe/yYwZt/H444/z7W//\nG5s3b+LcuUZOnTpJaelp/uVfnmHatBnOfisiIiL9qrOri1MVjRSdridjdCRm777rIS7XcPJL6vjN\nmkPMGRfFVxalOjuOiIjLeOSRx8nO/gvx8YmcOnWCX/7yf6itrWHy5KncccdSSktP873vPceMGbf1\nOK+ysoKf/OTnfP75Dtau/ZsKmoiIDHrtHV0cL2+goKSOgpI6CkvraW3rBOBkVSPfWDyyz17b5Qra\nqDgzMeF+bD1Qxuxx0cRG+Ds7kojIDfvLJ0Xszqvs1eeclBrGg/OSruuxI0emAeDvH8DRo7n8/e/Z\nGAxGGhrqL3ns2LHjAAgLC6OxsbH3AouIiAwQre2dHCutJ7+7kBWXNdDe0eW4PzLEhxExQaTEBHH7\n1DjOnW3psywuV9BMbkb+191j+P5vdvLmxgL+/bEJGAwGZ8cSEXEp7u7uAGzY8DENDQ384hf/Q0ND\nA9/4xuOXPNbNzc3xtc1m67eMIiIifaW5tYPC0/WOK2THyxvo7LL/jDMAw8L8SIkJcpSyAF8Px7k+\nXu4qaBcbPyKMCSkW9hVU8cWRCqamRTg7kojIDXlwXtJ1X+3qLUajkc7Ozh7H6urqiIyMwmg0snXr\nJ7S3t/drJhERkf7Q2NxOYUkd+d3/nKo4y5efORoNBmIj/B1lLDkmEF8vd6dldcmCBvDQvCQOFlfz\nl81FjEsOxcvDZd+KiEi/iI2NJz8/j8jIKIKCggCYM2cezz33bxw5cpglS+4iLCyM3//+t05OKiIi\ncmvqG1sdwxXzS+oorTrnuM/kZiApOpARw+2FLDEqcEAtPjhwktwgS5A3d0wZzrodJ3h/x0kemJPo\n7EgiIgOa2WwmO/uDHsciI6P4wx/ecdxesOAOACwWf6qqzpKQcP4qX0JCEv/937/pn7AiIiI3oLq+\nhfyS2u5CVk9FTZPjPg+TkZGxZkbEBDFieBDxkQF4uLtd5dmcy2ULGsDiabF8dric9btPcdvYSMKD\nfZwdSURERERE+pDNZqOytvn8FbJTdVQ3nJ8T5uXhxpiEEMcVsrgIf0xurrP9s0sXNE93Nx6al8xr\naw7zzqZC/t9l6c6OJCIiIiIivajLZqPceq7HkMX6xjbH/b5eJsYnh3ZfITMTE+aH0ei6iwi6dEED\nyBhhIXV4EDnF1RwstjI2MdTZkURERERE5CZ1ddkoqWy0L+hxqpbC0/U0Np9fxCrQ14PJI8NI6V7U\nIyrUF+MgWtXd5QuawWDg0fkprPz9Lt7eWMiouGCXuoQpIiIiIjKUdXR2kXeyhi8Oltk3hT5dR3Pr\n+VWHQwK8HEMWR8QEEWb2HtTbbLl8QQP7PgXzxg9j077TbNhTwh1TYp0dSUREBpFz586xYsUK6uvr\naW9v5+mnn+Y3vzm/YEplZSX33nsv3/rWtxzHXn31VdatW0d4eDgAd911F8uWLev37CIiA017RyfH\nyhq6r5DVUVxWT1v7+U2hw4N9mJQa6LhCFhro7cS0/W9QFDSAu2+L54ujFfz9sxNMS4sgyM/T2ZFE\nRGSQWL16NfHx8TzzzDNUVFTwxBNP8PHHHzvu/8Y3vsHdd999yXlf+cpXWL58eX9GFREZcFraOigq\n7d4U+lQdx8ob6Oi0Oe6PtviSnmxhuMWXlJigIf97/KApaH7e7tw3K4E//iOf97YU842lo5wdSUTE\nJT3wwJ18+OEH137gEGI2m8nPzwegoaEBs9nsuG/Hjh3ExcURGRnprHgiIgNKU0s7BafrKThlX9Dj\n5JmzdHXvCm0wwPDw85tCp8QE4eft7tjeRQZRQQOYlR7FlgOl7Dh8hjnjokkaFujsSCIiMggsWbKE\n7OxsMjMzaWho4Ne//rXjvj/+8Y9kZWVd9ryPP/6YTZs24eHhwX/8x38QExNz1dcxm30wmW59bx6L\nxf+Wn6O/uFJWcK28yto3lPVS9Y2tHD5WTe6xanKLqzleXk93H8PNaCBleBCjE0NJSwhhZFwwvt7u\nTs3bG/oy66AqaEajgccyU3j5z/t4c2MB3/tKhksvsSki0puefPIxfvjDVURERHDmTDn//u/PYLGE\n0dzcTEtLC//6r88yatRoZ8cckNauXUtUVBSvv/46eXl5ZGVlkZ2dTUVFBU1NTQwfPvySc2bPns3U\nqVOZNGkSH3zwAS+99FKPYnc5tbVNV73/erjSp9CulBVcK6+y9g1ltas922rfFLr7Cll59fm/u9xN\nRsfVsRExQSREB+J5wabQTY0tNDW2XPKcQ+17e7WCN6gKGkDysCCmpoXzeW4F2w+VMys9ytmRREQG\nhFmz5vLZZ59y//0Psm3bVmbNmktiYjKzZs1h797dvPnmH/jBD/6vs2MOSPv27WPmzJkApKamUllZ\nSWdnJ1u3bmXq1KmXPWfs2LGOr+fNm8dPfvKTfskqItKbbDYbVfUtFJz6cg+yWqrqzhcsT3c30uKD\nHaUsPjIAd5NWVL8Vg66gASybk8T+AivvbSkmY4QFH6/LX0YVEXGW7KL32V95qFefc3zYGO5LWnrF\n+2fNmst///fPuP/+B9m+fSvf/va/8s47f+Ltt/9Ee3s7Xl5evZpnMImNjSUnJ4eFCxdSWlqKr68v\nbm5uHDp0iLlz5172nJdeeolFixaRkZHBrl27SE5O7ufUIiI3zmazcaamyb4pdPcVstqzrY77fTxN\njEsKtV8hGx7E8HA/3IwqZL1pUBY0s78nS6fH8retx1iz/TiPzk9xdiQREadLSEikurqKiooznD17\nlm3bthAaGsb3vvef5OUd4b//+2fOjjhgPfTQQ2RlZbF8+XI6Ojp4/vnnAaiqqiIkJMTxuKqqKl59\n9VVefPFFli1bxsqVKzGZTBgMBl566SUnpRcRubIum43TlY32FRa7/2loOr8ptL+POxkjLI4FPYaF\n+Q2qTaEHokFZ0AAWTBrOtpxyPtlbyuz0KKItfs6OJCLicF/S0qte7eor06bN5De/+SW33Taburpa\nEhPtV3W2bt1MR0dHv+dxFb6+vrzyyiuXHP/Vr37V47bFYuHFF18EYMSIEbzzzjv9kk9E5Hp1dnVx\nqqKR/FPnC1lT6/m//83+nkwdFU5K96bQEcE+g3pT6IFo0BY0d5ORh+cn8/P3DvLWxkK+8/A4/eES\nkSFv9uy5fOtbT/LGG2/T0tLMSy+tZPPmjdx//4Ns3LieDz74u7MjiohIL2rv6OJ4eUP3/LE6ikrr\naW3rdNxvCfJiQkr3FbLhQVgCvfQ7s5MN2oIGkJ4YwpiEEA4dq2ZfQRUTR4Q5O5KIiFONHJnG1q1f\nOG6/+eZ7jq9nzpwNwJIld+Hr60tTk2uspiUiIud1ddk4Xt7A+r2lHMivoLisgfaOLsf9kSE+9gU9\nhgeRMiyI4ADNPx5oBnVBMxgMPDI/mSP/U8M7m4oYkxCCh/ut7y8jIiIiIjJQNLd2kHu8hpwiKweP\nVXO2ew6ZAYgJ83PMH0uJCSLA18O5YeWaBnVBA4gI9mHBpBg++uIUH31xirtnxjs7koiIiIjILamq\na+ZAkZWDRVbyTtXR2WXfGTrQ14NZ6ZHMHB9DZJAnvlrN3OUM+oIGsHR6HDsOn+HDz08yY0wEoYHe\nzo4kIiIiInLdurpsFJXWk1NsJaeomjLrOcd9seH+pCeFkJ4USmyEP0aDwaU2fpaehkRB8/Y0sWxu\nIv/z/lH+8kkR/3TvGGdHEhERERG5qqaWDg4fr7YPXSyu5lyLfbVFD5ORcUmhjE0KIT0xFLO/p5OT\nSm8aEgUNYGpaBJv3l7Inv4qjJ2oYGRfs7EgiIiIiIj1U1DSRU2TlQJGVwtP1jqGLZn9PJo0MJz0x\nhJGxZq2rMIgNmYJmNBh4dH4KL/1hD29tLOT5Jydp13MRERERcarOri6KTtdzoMg+dPFMTZPjvvjI\nANKTQhiXFEpMmJ+Wvx8ihkxBA/sf8tvSI/k0p5zN+0qZnxHj7EgiIiIiMsQ0Nrdz+Fg1OcXVHCqu\ndmwU7enuxvjkUPvwxcQQAv00dHEoGlIFDeC+WYnszqtizbbjTB4VToCPlhoVERERkb5js9k4U9Pk\nuEpWdLqeLpt96GJIgCdT08JJTwoldXgQ7iYNXRzqhlxBC/D14J6Z8by9qZDVnx7jiUWpzo4kIiIi\nIoNMR2cXBSV15BTZF/morGsG7HuTJUQHkJ5ov1IWbfHV0EXpYcgVNIC5E6L5NKeMTw+UMWdcNLER\n/s6OJCIiIiIu7mxTG4eOVXOgqJrc49U0t3YC4OXhRsYIC+lJoYxJCNFm0XJVQ7KgmdyMPDI/mZ+8\nc4A3NxTw78sn6JMLEREREbkhNpuNMus5+9DF4mqKS+vpHrlIaKAXM0ZHkp4cyoiYIExuWpxOrs+Q\nLGgAo+KCmTjCwt78Kj7PrWDa6AhnRxIRERGRAa69o4v8klpyCqvJKbZirW8BwGCApOjA7v3JQokK\n8dEFALkpQ7agATw0N4mDxdX8ZUsR45JD8fYc0t8OEREREbmMhnNt5BRbySupZ19+Ja1t9qGL3p4m\nJo8MIz0xlDGJIfh5uzs5qQwGQ7qRhAZ5c8eU4fz9sxO8v/MEy+YkOTuSiIiIiDiZzWajpLKRnGL7\nAh/HyxroHrlImNmbcemhpCeFkjwsUEMXpdddV0H74Q9/SE5ODgaDgaysLMaOHQtARUUF3/nOdxyP\nKykp4ZlnnuHOO+/sm7R94I6psXx2qJz1u0qYNTaK8GAfZ0cSERERkX7W3tHJ0ZN15BRZySm2UtPQ\nCoDRYCAlJoj0pFDmTh6Op0YtSh+7ZkHbtWsXJ0+e5N1336W4uJisrCzeffddAMLDw/nTn/4EQEdH\nB48//jjz5s3r28S9zNPdjYfmJfPLNYd5e1Mh/3tZurMjiYiIiEg/qGts5WBxNQcKrRw5WUNbexcA\nvl4mpo6y7002OiEYXy/70EWLxZ+qqrPOjCxDwDUL2s6dO5k/fz4AiYmJ1NfX09jYiJ+fX4/HrV69\nmoULF+Lr69s3SfvQxBEWUocHcbD7MnZ6UqizI4mIiIhIL7PZbJyqaOzeMNrKiTPny1ZkiA/pSaGk\nJ4aQNCwQN6OGLopzXLOgWa1W0tLSHLeDg4Opqqq6pKD99a9/5Xe/+13vJ+wHBoOBRzNTeP53u3ln\nUyGj4oJxN+l/ShERERFX19reydETteQU20tZXWMbAG5GAyNjzfZSlhRCuFnTXGRguOFFQmxfbu5w\ngf3795OQkHBJabscs9kHk8ntRl/2EhZL724ubbH4s2RmPOu2HWPn0Urun5fc68/vKpS1b7hSVnCt\nvMraN1wpq4jIhWoaWuxDF4usHD1ZS3uHfeiin7c700dHkJ4USlpcMD5eQ3q9PBmgrvmnMiwsDKvV\n6rhdWVmJxWLp8ZgtW7Ywbdq063rB2tqmG4x4qb4a/7tgYjSb95Tw9oZ8xsSZMft79srzutJ4ZWXt\nG66UFVwrr7L2jd7KqpInIv2hy2bjRPlZ+wIfRVZOVTY67ou2+JKeGMq4pFASogIwGrXKhwxs1yxo\nM2bM4NVXX+Xhhx8mNzeXsLCwS66UHTp0iMWLF/dZyP7i6+XO/bMT+MPH+by3pYj/dWfatU8SERER\nkX7X0tbBkRO1HCiycrC4moZz9qGLJjcDafHB9g2jE0OwBHk7OanIjblmQZswYQJpaWk8/PDDGAwG\nVq5cSXZ2Nv7+/mRmZgJQVVVFSEhIn4ftD7eNjWLL/jJ25lYwZ3w0ycOCnB1JRESc7Ny5c6xYsYL6\n+nra29t5+umn+c1vfkNTUxM+PvZ5KytWrGD06NGOc9rb23nuuecoKyvDzc2Nl19+mZiYGGe9BZFB\nwVrfTE5RtX3T6JN1dHTahy4G+Lgzc0wk6UkhjIoLxttTQxfFdV3Xn94L9zoDSE1N7XF73bp1vZfI\nyYxGA49mJvPyn/fx1oZCvvdEhi6Fi4gMcatXryY+Pp5nnnmGiooKnnjiCSwWCy+//DIpKSmXPef9\n998nICCAVatWsX37dlatWsXPfvazfk4u4tq6umwcK29wDF08XXXOcV9MmJ9jgY/4yACMBv2+JoOD\nPl64jORhQUxLC2dnbgXbDpYxe1y0syOJiIgTmc1m8vPzAWhoaMBsNl/znJ07d3LPPfcAMH36dLKy\nsvo0o8hg0dzaQe7xGnKKrBw8Vs3ZpnYATG5GxiaGkJ4YQnpSKMEBXk5OKtI3VNCu4IE5SewrsPK3\nrcfISA1zbFAoIiJDz5IlS8jOziYzM5OGhgZ+/etfs2rVKn7+859TW1tLYmIiWVlZeHmd/4XRarUS\nHBwMgNFoxGAw0NbWhoeHh7PehsiAVVnX7LhKln+qjs4u+6rhgb4ezEqPJD0plFGxwXh63PpK4CID\nnQraFZj9PblzRhzvbSlm7bbjPJp5+SEsIiIy+K1du5aoqChef/118vLyyMrK4qmnnmLEiBEMHz6c\nlStX8uabb/L1r3/9is9xuW1qLjZQt6LpS66UFVwr70DPWtvQwoc7TvDZwTJKKs6vGps4LJDJoyKY\nNCqcxOigATfVZKB/Xy/kSlnBtfL2ZVYVtKvIzIhhW04Zn+wrZda4KIZZrr3Pm4iIDD779u1j5syZ\ngH0edmVlJfPmzcPNzV6m5s2bx4cfftjjnLCwMKqqqkhNTaW9vR2bzXbNq2cDeSuavuBKWcG18g7k\nrKcqzrJhdwlfHK2go9OGh7ubfcXFpBDSE0N7bHNUXd14lWfqfwP5+3oxV8oKrpW3N7JereAZb+mZ\nBzl3k5FH5ifTZbPx9sbC6/r0U0REBp/Y2FhycnIAKC0txcfHh69//es0NDQA8MUXX5CcnNzjnBkz\nZvDxxx8DsHnzZqZMmdK/oUUGkC6bjYPFVv7v2/t5/ve7+ezwGUICvXl84QjefGER//LAWOaMi+61\nPWhFXJmuoF3D2ET7HhoHi6vZm19FRmqYsyOJiEg/e+ihh8jKymL58uV0dHTwwgsvUFtby1e/+lW8\nvb0JDw/nn//5nwF46qmneO2111i8eDE7duzgkUcewcPDgx/96EdOfhci/a+1vZOdh8+wfncJZ2rs\nV4hHxppZMCmGMYkhGA0GvDxNuMZ1E5H+oYJ2HR65PZnc4zW8+0khYxJD8HTXBFURkaHE19eXV155\n5ZLjixcvvuTYa6+9BuDY+0xkKKprbOWTfafZsr+MxuZ23IwGZoyOIHNSDMPDXWeekYgzqKBdh/Bg\nHxZMjuGjz0/x0ecnuee2BGdHEhERERlwTlWcZf3uEr44UkFnlw0/b3eWTo9j3oRogvw0fFHkeqig\nXael0+LYcfgMH31xipljIgkN8nZ2JBERERGns88vq2b9rlPknaoDIDLEh8yMGKaNjtDII5EbpIJ2\nnbw9TTw4J4nfvn+EdzcX8fS9Y5wdSURERMRpWts72XGonPV7TlPRPb9sVJx9ftnoBPv8MhG5cSpo\nN2BqWjib95eyN7+KIydqGBUX7OxIIiIiIv2q9uyX88tKOdfSgcnNwIwxESyYNJyYMG1JJHKrVNBu\ngMFg4LHMFF58Yzdvbyxk5dcmYXLTTgUiIiIy+J08Y59ftuvo+flld3bPLwvU/DKRXqOCdoNiI/yZ\nNS6KrQfK2Ly/lMyMGGdHEhEREekTXTYbB4uqWb+75/yyBZNimJYWgYfml4n0OhW0m3DvrAR2H61k\nzbbjTBkVToCPh7MjiYiIiPSa1rZOPjtczobdJVTUNgOQFmdmweThpMUHa36ZSB9SQbsJAT4e3HNb\nPG9tLCR7azFfvWOksyOJiIiI3LLas61s2nuarQfOzy+bOTaSBRkxDNP8MpF+oYJ2k+ZOiGbrgTK2\n5ZQze1w08ZEBzo4kIiIiclNOnjnLP3afYvfRSjq7bPj7uHPXjDjmThhGoK9GCon0JxW0m+RmNPLo\n/GT+7zsHeGtjAf++fKIu94uIiIjL6OqykVNkZf3uEvJL7PPLokJ9WTAphqmjwjW/TMRJVNBuwci4\nYDJGWNiTX8XnuWeYPjrS2ZFERERErqqlrYPPDp1hw54SKr+cXxYfzMJJMaTFB2PQB84iTqWCdose\nnJfEweJq/rq5mPHJFrw99S0VERGRgaemoYVN+06zdX8ZTa0dmNyM3DY2ksxJMQyzaH6ZyEChNnGL\nQgO9WTw1ljXbj/P+jhMsm5vk7EgiIiIiDsfLG9iwu4Tdeefnl909M56546MJ0PwykQFHBa0XLJoy\nnG0Hy1m/u4Tb0qOICPZxdiQREREZwjq7bOwrqGL9rlMUnK4HIPrL+WVp4bibNL9MZKBSQesFHu5u\nPHx7Er9YfZh3NhXyv5elOzuSiIiIDEEtbR1sP1jO5v1llFefA2B0QjALJsWQFqf5ZSKuQAWtl0xI\nsTAy1szB4mpyiqykJ4U6O5KIiIgMETUNLWzce5qtB8pobu3A3WRkVnokmRkxRGt+mYhLUUHrJQaD\ngUfnJ7Pyd7t5e1Mho+KCcTczlFj6AAAgAElEQVQZnR1LREREBrHj5Q38Y9cp9uRV0WWzEeDjzsKZ\n8dw/fwTtLW3OjiciN0EFrRdFW/yYNzGajXtOs373KZZMi3N2JBERERlkurps7C+0sn73KQq/nF9m\nOb9/mbvJjSB/T6pU0ERckgpaL7tnZjyf51bw/o6TTB8didnf09mRREREZBBobu1g+6FyNu4poaqu\nBYAxCSEsmBzDqFiz5peJDBIqaL3Mx8udB+Yk8sZHefx1SxHfvDPN2ZFERETEhVXXt7Bp72m25pyf\nXzZ7XBSZGTFEhfo6O56I9DIVtD4wc2wkW/aX8nluBXPHR2Ox+Ds7koiIiLiYY2UNrN99wfwyXw8W\nTY5n9vhoAny0f5nIYKWC1geMBgOPZqbwwz/t5c0NBUxJH+bsSCIiIuICur7cv2x3CUWl9vllwyx+\nLJgUw5RR4VqATGQIUEHrI0nRgUwfHcGOw2fY8MVJJiaFODuSiIiIDFDNrR1sO2ifX2att88vG5sY\nwoJJMYzU/DKRIUUFrQ89MCeRvQVV/PHDo6R+cwq+Xu7OjiQiIiIDiLW+mY17TrPtYBnNrZ24m4zM\nGRdF5qQYIkM0v0yurbOrk5bOVpo7WmjuaKGlo5mWzlY8mgzUNzT3eKzNZut5G9sFXzse1PP2Bbcc\nj7ddeM/Fj7/gK1vPYxc+58XP5VvrQWNj6+Xz2S59dttFCe0Pu/L7O/9yV3l/F+W90uNnuWXgj/mS\n1+8tKmh9KMjPk7tmxPHXzcWs2XacxzJTnB1JRERuwrlz51ixYgX19fW0t7fz9NNPY7FYePHFFzEa\njQQEBLBq1Sq8vb0d52RnZ/PKK68wfPhwAKZPn85TTz3lrLcgA0xxaT3rd5ewN98+vyzQ14NFU2KZ\nMy4Kf80vGzLaO9tp7vyyWPX8d3PnZY51tDjKWEtHM80dLbR1tTv7bQw51vYqnkh5tM+eXwWtj2Vm\nxLDj8Bk27ytldnoUw8L8nB1JRERu0OrVq4mPj+eZZ56hoqKCJ554gtDQUJ577jnGjh3Lj3/8Y7Kz\ns3nsscd6nLd48WJWrFjhpNQy0HR2dbGvwL5/WXFpAwAxYfb5ZZNHan6ZK7HZbLR+WZQuunp16bEL\n/n1BGWvpaKHD1nnDr20yuOFl8sLL5EWAZwDebl54d9/2Mtm/9jZ5ERIYQFNj9154F4yQNThuGC6+\nCww9jxm4dGjt5YbbGi5+rgsec/FXVzo/IMCLsw0tV8xyua+uluXyea/y/gyXP//CY18+1fi4VM7V\nd1zyuN6igtbHTG5GvnH3GF74n895a2MBzz4yXuPIRURcjNlsJj8/H4CGhgbMZjO/+tWv8POzf+gW\nHBxMXV2dMyPKANbc2sG2nDI27DlNdfcvoOnd88tSNb+s3305JLDnValLr1R9eQXrwmNttjbOtTXT\n0tFy2SF21+Lh5oG3mye+7r6Eeoc4ipW3m+cF5cobLzfP8/eZvC54nBfubtc3ZcZi8aeq6uwNZ3QW\nV8rr4+HNOfouqwpaP8gYGU56Ygg5xdXsya9iUmqYsyOJiMgNWLJkCdnZ2WRmZtLQ0MCvf/1rRzlr\nampi7dq1vPLKK5ect2vXLr7+9a/T0dHBihUrGDVqVH9HFyey1jWzce9pPs0po6WtEw+TkTnjo8nM\nGKb5ZTepvbO9++pU82WG/LU4jrd0tPYYInjhFay2zrYbfl0DBrxMnvh6+GD2DMTbN7znVasLrmI5\nCtVFx7zcPHEzuvXBd0UGGxW0fvLw/GRyT9Tw7ieFjE0MwdNd/4OKiLiKtWvXEhUVxeuvv05eXh5Z\nWVlkZ2fT1NTEU089xZNPPkliYmKPc9LT0wkODmbOnDns37+fFStWsG7duqu+jtnsg8l06z8fXGn/\nTVfKCteXN+9EDWu2FrPzUBldNggO8GTZ7SksmhZHgG//zS8byN/b5vYWKs9ZqTxXTU7xWZram2lu\nb6GpvbnHPz2PtdDRdePDytyMbvi4e+Pj4U2weyA+7t54u3vj4+5lP37J197djzl/zMvk6ZJXOgfy\nn4HLcaW8fZlVBa2fhJt9WDh5OB/sPMlHn5/kntsSnB1JRESu0759+5g5cyYAqampVFZW0tbWxj/9\n0z+xdOlS7rvvvkvOSUxMdJS28ePHU1NTQ2dnJ25uVy5gtbVNt5zVlYYJuVJWuHrezq4u9uZXsWF3\nCcVl9vllw8P8WDDZPr/M5GaktamVqqbWy57fn1n7Q3tnOzUttVhbaqlpqaG6uRZrSw3VzTVUt9Rw\nrv36/qx7GN27r0j5EOwZ7Bj6d8mcq4uvVl1w3GQ03Xi5sgFt0NUGjbTTiH0hDmd/X2+EK2UF18rb\nG1mvVvBU0PrRkmmxfHaonA8/P8WMMZFYgryvfZKIiDhdbGwsOTk5LFy4kNLSUnx9fXn99deZPHky\ny5Ytu+w5v/3tb4mMjGTp0qUUFBQQHBx81XImrqmppYNPc8rYtLeE6gZ7+RqXFMqCSTGMGB7kkldd\nrkeXrYvalnqqLyhd1uZax+36tobLnmcymgjxMhPrH0OIdzAhXmaGhYbR3mTrLlbeFwwR1JBAGZpU\n0PqRl4eJB+cm8Zt1R/jLJ0U8fd8YZ0cSEZHr8NBDD5GVlcXy5cvp6Ojg+eef59lnn2XYsGHs3LkT\ngClTpvDtb3+bp556itdee40777yTZ599lnfeeYeOjg5+8IMfOPldSG+qqmtmw54Sth0sp7V7ftnc\nCdFkZsQQEezj7Hi3zGazcba90V6+mmuwttQ6ilh1cw01rXV02bouOc+AAbNXEMlBCYR4BxPqFdxd\nxIIJ8TYT4OGP0dBztUpXunIi0h9U0PrZlFHhfLK/lL0FVeSeqCEtLtjZkURE5Bp8fX0vWQRk+/bt\nl33sa6+9BkBERAR/+tOf+jyb9B+bzUbh6TrW7y5hX0EVNhsE+XmwdFoss8dF4+d9favrDRTNHc09\nrnp9+W9rSy01zTVX3F/L38OPWP9hPYpXiFcwod7BmD2DdNVL5BapoPUzg8HAY/NTePGN3by9sZDn\nvzYJk5v2PRERERmobDYbu/Mq+WT/PgpO2bdTiA33Z8GkGCaNDBuwP8fbOtupaanB2lxD9YVXwLq/\nbupovux53iZvwn0sFxQw+1DEUO9ggr3MeLhpI22RvqSC5gSxEf7MHhfFlgNlbN5XSuakGGdHEhER\nkSv4ZF8pb24owGCwzy9bODmGlBjnzy/r7OqktrX+oqtf9gU5attqqWu5/Dwwd6M7IV5m4gNjHVfA\nLhyK6OOuOfIizqSC5iT3zkpgd14la7YfZ8qo8H5ddldERESuT3NrB3//7DheHm789H/PxqsfL5bZ\nbDYa2s52L8BhL14XDkesba2/7Dwwo8FIqI+ZFHMSoV7mi66EBRPg4ef0cikiV6aC5iT+Ph7cc1sC\nb24o4G9bi/na4pHOjiQiIiIXWb+7hLNN7dwzM56Y8N5dzMJms9HU0XzBla/u4YfdV8FqWmpov8K+\nX4EeAcQFDCfkggIW2j0XLMgzkIjwIC28IeKiVNCcaM74KLYeKGX7wXLmjI8mPjLA2ZFERESkW8O5\nNj7edQp/H/ebno7Q2tl2wRDE2kuGIrZ0tlz2PF+TD5G+4T2ufNlXRTQT7GXG3c21FiQRket3XQXt\nhz/8ITk5ORgMBrKyshg7dqzjvvLycv7t3/6N9vZ2Ro0axYsvvthnYQcbN6ORR+en8H/e3s9bGwr4\n98cnYtSQAxERkQHh/R0naG3r5P5ZCXh7Xv5Xps6uTmpa6i4qXucX4jjb3njZ8zyM7t3FK95RvC4s\nYt4mr758ayIygF2zoO3atYuTJ0/y7rvvUlxcTFZWFu+++67j/h/96Ec8+eSTZGZm8sILL1BWVkZU\nVFSfhh5MUmPNTEoNY3deJTsPn2HGmEhnRxIRERnyrHXNbN5fSmigFzPTwzlWf5Kj585xvLLUcSXM\n2lxDXWs9NmyXnO9mcMPsFUS0X6RjGfrzQxGD8XP31TwwEbmsaxa0nTt3Mn/+fAASExOpr6+nsbER\nPz8/urq62Lt3Lz/96U8BWLlyZd+mHaQenJtETpGV97YUMyHFcsVP6URERKR//GXbEQguIXhkC9/d\n8REtna097jdgINAzgITAOEK7l6G/sIAFegZcsiGziMj1uGYTsFqtpKWlOW4HBwdTVVWFn58fNTU1\n+Pr68vLLL5Obm0tGRgbPPPNMnwYejEICvVg8LZY1246zbscJHpyb5OxIIiIiQ0qXrYuSs6Ucrs5j\n/5lcyn3L8EiAklaweIcwJSSDlPBYPDt8CPE2Y/Yy427UB6oi0vtu+G8Wm83W4+uKigq+8pWvEB0d\nzTe/+U22bNnCnDlzrni+2eyDyXTrO8xbLP63/Bz96Vp5ly9JY0duBRv3lHD3nCSGhTnv/bnS91ZZ\n+44r5VXWvuFKWUVuRktHC3k1hRyuziO3Oo+Gtu5VD20GOhuCmRmXTuaIDMJ9LID9/wmtjCgife2a\nBS0sLAyr1eq4XVlZicVi/4vKbDYTFRXF8OHDAZg2bRqFhYVXLWi1tU23GNn1/oK83rzLZifyi9WH\n+MVfD/Cvy9KdMjbdlb63ytp3XCmvsvaN3sqqkicDTWWTldzqPA5bj1JYd4xOWycA/u5+TI3IIJgY\n/vZBAylRFh4bN17zxESk312zoM2YMYNXX32Vhx9+mNzcXMLCwvDz87OfbDIRExPDiRMniIuLIzc3\nlyVLlvR56MFqQkooo+LMHD5WQ05RNeOSQ50dSURExKV1dHVQXHeCw9VHOVx9lMqm8x86x/hHMzpk\nJKNDUxnuPwwDBl7+8z7odGfZnESVMxFximsWtAkTJpCWlsbDDz+MwWBg5cqVZGdn4+/vT2ZmJllZ\nWTz33HPYbDZSUlKYN29ef+QelAwGA4/OT2Hl73bx9qYC0uLNuPfCcFAREZGhpKHtLLnV+eRaj3K0\npsCxwIeHmwfpoWmMDh3JqJARBHkG9jhvf2EVRaX1jE8OJTE68HJPLSLS565rDtp3vvOdHrdTU1Md\nX8fGxvL222/3bqohLCrUl9snDmP97hLW7y5hybQ4Z0cSEREZ0LpsXZw+W9Z9lSyPUw2nHUvfh3oF\nMzUyg9EhI0kyJ1xxYY+uLhvZW49hMMB9sxP7M76ISA9afmgAumtGPJ/nnmHdjhNMS4sgOECbVYqI\niFyopaOFvNoicq1Hya3Oo757gQ+jwUhyUAJpoamMDhlJuI/luoYq7jh8hlLrOWaOiSQ61Lev44uI\nXJEK2gDk42Xi/tmJ/P6jPN7bUsw370q79kkiIiKDXFVTtf0qmfUoRXXH6Ohe4MPP3ZcpERMZHTqS\nkcHJeJu8b+h52zs6WbP9GCY3I/fcFt8X0UVErpsK2gA1Y2wkm/eX8vmRCuaMjyYlJsjZkURERPpV\nZ1cnxfXHOWzN43D1USqaqhz3xfhFMTp0JGkhI4kNGHZLm0Jv3ldKTUMrCyfHaNSKiDidCtoAZTQY\neCwzhR/8aS9vbSjg+1+dhNGo1aRERGRwO9vWaF8GvzqPo9UFtHS2APYFPsaGpjE6NJW0kNRLFvi4\nWU0tHby/8yTenm6a9y0iA4IK2gCWGB3IjNERfHb4DJ/mlDFnfLSzI4mIiPQqm83G6cYyDlvtC3yc\nbChxLPAR4hXMlMiJjLnGAh+34uNdp2hsbufeWQn4ebv3+vOLiNwoFbQB7oE5iewtqCL702NkpIbp\nh4eIiLi8lo5W8muLKDpRxN7Th6hvawDsC3wkBcUzOnQko0NSCfcJ69O9yOobW1m/+xSBvh4syIjp\ns9cREbkRKmgDXKCfJ3fNiOcvm4tYs+0YyxeMcHYkERGRG2ZtrnbMJSusLb5kgY+0kFRGBqfg435j\nC3zcinU7TtDW3sVDc+Pw9NC+oyIyMKiguYD5GcP4NKeMzftLmT0umpgwP2dHEhERuSr7Ah8nOFx9\nlFxrHmeaKh33Dete4GNm4gQCu0JuaYGPm1VZ28TWA2WEmb25LT2q319fRORKVNBcgMnNyCPzk/mv\nv+Tw1oYC/r9Hx/fpkA8REZGbcbatkSPV+RyuPsrRmgKaO7oX+DC6MyZ0FKND7At8mL3sKxNbQv2p\nqjrrlKyrtx2ns8vGfbMSMLn1f0EUEbkSFTQXMSYhhHFJoRwosrI7r5LJI8OdHUlEZMg4d+4cK1as\noL6+nvb2dp5++mksFgvPP/88ACNGjOCFF17ocU57ezvPPfccZWVluLm58fLLLxMTM7jmOdkX+Cjn\nsPUoudVHOXHRAh+TIyaQFjKSlKAE3N0Gzhzqk2fO8sWRCoaH+5GRGubsOCIiPaiguZCHb0/i8PFq\n/rK5iPTEUI2XFxHpJ6tXryY+Pp5nnnmGiooKnnjiCSwWC1lZWYwdO5ZnnnmGrVu3Mnv2bMc577//\nPgEBAaxatYrt27ezatUqfvaznznxXfSO1s428msK7UMXq/Opa60Hzi/wkRaSyujQkUT08QIft+Jv\nW4sB+0JcxgGaUUSGLhU0FxJm9mHh5OF8sPMkH35+kntnJTg7kojIkGA2m8nPzwegoaGBoKAgSktL\nGTt2LABz585l586dPQrazp07ueeeewCYPn06WVlZ/R+8l1ibazhcfZTD1qMU1h2jo6sDAF93HyaF\nT2BM6JcLfPg4Oem1HT1Zy+HjNYyMNZMWF+zsOCIil1BBczFLpsWy4/AZPvriFDPHRmIJ6r/VrkRE\nhqolS5aQnZ1NZmYmDQ0NvPbaa7z44ouO+0NCQqiqqupxjtVqJTjYXgCMRiMGg4G2tjY8PDz6NfvN\n6Ozq5Fj9CQ53bxh95lyF475ov0hGh4xkdOhI4gJinLLAx82y2Wy8t+X81bOBeoVPRIY2FTQX4+Vh\nYtncRH7z9yO8+0kR375vjLMjiYgMemvXriUqKorXX3+dvLw8nn76afz9/R3322y2az7H9TzGbPbB\nZLr14esWi/+1H3SRhtZGDpTnsq/sEDlnjnCuvRkADzd3JkaNYULkGMZHpRHq07tXnW4m683acbCM\n4+UNTB8byeSx0Tf1HP2Z91Ypa99Q1r7jSnn7MqsKmguaMjKcLftK2VdQRe7xGtLiNURDRKQv7du3\nj5kzZwKQmppKa2srHR0djvsrKioIC+u52ERYWBhVVVWkpqbS3t6OzWa75tWz2tqmW85qsVzfyog2\nm43SxvLuoYt5nGg4dcECH2Ymho1ndGgqyUGJeHQv8GE7B1Xnem/VxevN2hs6u7r4/bpcjAYDS6YM\nv6nX7c+8t0pZ+4ay9h1XytsbWa9W8FTQXJDBYODRzBReeGM3b20s4IUnJ2uJYBGRPhQbG0tOTg4L\nFy6ktLQUX19foqOj2bNnDxkZGaxfv57HH3+8xzkzZszg448/5rbbbmPz5s1MmTLFSenPa+tsI7+2\niEPWo+RW5/VY4CMxKI7RISNJC0kl0jd80A3/++zQGc7UNDErPYrIEF9nxxERuSIVNBc1PNyfOeOi\n2by/lE/2nmbB5OHOjiQiMmg99NBDZGVlsXz5cjo6Onj++eexWCx8//vfp6uri/T0dKZPnw7AU089\nxWuvvcbixYvZsWMHjzzyCB4eHvzoRz9ySvbq5pruuWRHKagtPr/Ah8mHSeHjGR06klEussDHzWpr\n72Tt9uO4m4zcPTPe2XFERK5KBc2F3TsrgV1HK1j72XGmpEUQ6DvwJ56LiLgiX19fXnnllUuOv/XW\nW5cce+211wAce5/1t86uTgprj5HbXcrKL1rgIy0klTGhI4kLGO5SC3zcik37TlN7tpU7pg7H7O/p\n7DgiIlelgubC/LzduXdWAn9eX8Dfthbz5OKRzo4kIiJO9MmpT/l42ybHAh/uRvfuFRdTSQtJJdjL\n7OSE/a+ppZ0Pd57Ex9PE4qmxzo4jInJNLlfQSs6W8b2dbxDqGUKKOZFkcyKxATG4G13urfSK2eOi\n2LK/jO0Hy5kzLpqEqABnRxIRESepaLbi5+HLxLBxpIWkkmJOcizwMVR9+PkpzrV08MCcRHy9hvb3\nQkRcg8u1Gm+TF2avQApqiymoK4bj9k8IEwJjSQ5KJMWcSGzAMExDpLC5GY08lpnMj9/az1sbC8h6\nfCLGQTaxW0RErs8jI+5zqZXQ+lrt2VY27ikhyM+D2ycOc3YcEZHr4nItJtQ7mJcXPMfxsjMU1R2n\nsLaYwrpj5NcWkV9b5ChsiYFxJJsTSTEnEOsfg5vx1veVGahGDDczeWQYu45WsvPwGWaMiXR2JBER\nEadb99lx2jq6eGRmPJ7ug/f3ABEZXFyuoH3Jz92XcZbRjLOMBqCx7RxFdccoqCumsPYYebWF5NUW\nAuBhdCcxKJ7koARSzIkM9x826Arbg3OTOFBk5a9bipmQYsHb02X/04qIiNyyMzVNfJpTTniwDzPH\n6oNLEXEdg+a3eD8PX8aFjWFc2BgAzrY1Ulh3jMJae2k7WlPA0ZoCADzdPEgMjCfZnEByUCLD/aNd\nvrAFB3ixZFocqz89xrrPTvDgvCRnRxIREXGa7E+P0WWzcf+sBNyMQ2O1ShEZHAZNQbuYv4cfE8LG\nMiFsLAANbWcpqjtOQW0xhbXFHKnJ50hNPtBd2ILiSemewzbML8olC9uiyTFsyyljw54SbkuP1Eac\nIiIyJB0vb2BPXiXxkf5MHGFxdhwRkRsyaAvaxQI8/HsUtvrWsxTVFVNQd8xe2KrzOVJtL2xebl4k\nBXXPYQtKZJh/lEvsFeNucuOR25N5NfsQb20s5N8eTMegBUNERGSI+dvWYgAemJ2on4Mi4nKGTEG7\nWKCnPxPDxzExfBwA9a0NFNZ2F7a6Yg5X53G4Og/4srDFdy/rn8Awv4Fb2MYlh5IWH0zu8RoOFFkZ\nn6xPDkVEZOjIPVHDkRO1pMUHMzIu2NlxRERu2JAtaBcL9AwgI2I8GRHjAahrrbfPX6st7i5sRzlc\nfRQAb5O3vbAFJZBsTiLaL2LAFDaDwcAjtyez8ne7eGdTIaPjg3E3ud5wTRERkRvVZbPx3pbzV89E\nRFyRCtoVBHkGMiliPJO6C1ttS133oiP2q2yHrEc4ZD0CgI/Jm6TuFSKTgxKIcnJhiwr15faJw1i/\nu4R/7Cph6fQ4p2URERHpL3vyKjl55iyTR4YRG+Hv7DgiIjdFBe06mb2CmBwxgckREwCoaal1rBBZ\nWHuMg9ZcDlpzAfA1+ZBkTnAs6x8S2v+Lddw1I57Pj1Tw/s4TTB8dQXCAV79nEBER6S8dnV1kf3oM\nN6OBe2clODuOiMhNU0G7ScFeZqZETmRK5EQAqptrKewuawV1xeRUHSan6jAA/gd8u5f1ty86Eukb\n3ueTln28TDwwO5HffXiUv24p5v+5K61PX09ERMSZth8sp7K2mbnjowk3+zg7jojITVNB6yUh3mZC\nvDOYGpkBQHVzTff8tWMUNRzjQNVhDnQXNj93X5KDEuyFzZxIhE9YnxS26WMi2Ly/lC+OVDB3fDQp\nMUG9/hoiIiLO1treydrPjuPhbuTOGXHOjiMicktU0PpIiHcw07yDmRY1idBQP/JKTlJQa18hsqC2\nmP1Vh9hfdQgAf3c/x6bZKeYEwnupsBkNBh7LTOGlP+7hzQ0FrPzqJIxGLTcsIiKDy8Y9JdQ3trFk\nWixBfp7OjiMicktU0PqBwWAg1DuEUO8QpkdNwmazYW2uoaCuyLFS5L7Kg+yrPAjYN9lOCbIv6Z8S\nlEiYj+WmC1tCVAAzx0Sy/VA5Ww+UMnfCsN58ayIiIk7V2NzOh5+fwtfLxB1TYp0dR0TklqmgOYHB\nYMDiE4LFJ4QZUVOw2WxUNVsvWHSkmL2VOeytzAEg0MOf5O4VIpPNiYR5h95QYbt/TiJ7CyrJ/vQY\nk0aG4+ft3ldvTUREpF99+PlJmls7eHBuEj5e+rVGRFyf/iYbAAwGA2E+FsJ8LMyIthe2ymarfQ5b\nbTEFdcXsqTjAnooDAAR6BDiuriWbE7F4h1y1sAX6enDXjHje/aSI1duO8fiCEf311kRERPpMTUML\nm/aexuzvye0To50dR0SkV6igDUAGg4FwHwvhPhZui56KzWajoqnKMX+tsPZYj8IW5BnomL+WHJRI\nqHfwJYXt9onD+DSnjC37S5mdHsXwcO0PIyIirm3t9uO0d3Rxz8x43E1uzo4jItIrVNBcgMFgIMI3\njAjfMG6LntZd2Cop6N40u7C2mN0V+9hdsQ8As2fQBYuOJBLiZcbkZuSR+cn89N0c3tpYyIpHx/f5\nUv8iIiJ9pcx6ju2HyokM8WH6mAhnxxER6TUqaC7IXtjCifANZ9aw6dhsNsrPVVBYd6x7af9idp3Z\nx64z5wtbitk+HDJthBe5+XXszqtk8shwJ78TERGRm5P96TFsNrh/diJuRqOz44iI9BoVtEHAYDAQ\n5RdBlF8Es4dNp8vWZS9s3cv6F9Ye44sze/nizF4IBK90b/58NJc2v6mMsiQR7GV29lsQERG5bsVl\n9ewrqCIxOoDxyaHOjiMi0qtU0AYho8FItF8k0X6RzImZ4ShsXy46kmstosOzhLcLS6AQQr2CHatE\nZniNwmDzxGjQp5EiIjLw2Gw2/ralGIAHZidquL6IDDoqaEPAhYVtbsxMmlvbyfrTBprdKxg12sap\ncyfZWb6bneW7+eNRcDeaiPANJ9I3nCjfCCJ9w4n0jSDYK0g/CEVExKkOH68h71QdYxJCGDFcI0BE\nZPBRQRuCvD3deWTGRH61NheDXyj/576vUtpYTkFtMVXtlRyvPk35uQpKzpb2OM/TzYPI7sIW1V3a\nIv3CCfQIUHETkUHtr3/9K3//+98dt3NyckhPT3fcrqys5N577+Vb3/qW49irr77KunXrCA+3z/e9\n6667WLZsWf+FHoS6bDbe21KMAbh/doKz44iI9AkVtCFqUmoYn+wrZX+hlSPHaxmdEE2MfzQWiz9V\nVWfpsnVR1VxN+bkKyhvPUH6ugrJzZzh19jQnGk71eC5vk3d3YQsn0i/CUd78Pfyc9O5ERHrXsmXL\nHOVq165dfPTRR6xcuTOD1fwAACAASURBVNJx/ze+8Q3uvvvuS877yle+wvLly/st52C360gFJZWN\nTE0L13YxIjJoqaANUQaDgUfnJ/PCG7t5a2MhL37dvhT/l4wGo2MvtnGW0Y7jHV0dVDZZ7cXt3Pni\ndqz+JMX1J3q8hp+7r32IpF94jytvPu4+/fU2ReQGddm6aGg7i7GpA/2IuLxf/OIX/OQnP3Hc3rFj\nB3FxcURGRjox1eDX0dnF6m3HcDMauPc2XT0TkcHrun76/vCHPyQnJweDwUDW/9/encdHXd9733/N\nTDLZt0ky2XfWhNWwBCJhURQRLaBYvGptTz1d1B7v02P70FI99PRy7a29bdXSqlc3b6oooqJUcWMR\nCAk7JCyBJCwJyWRPCEkgy1x/BAfCFpYkM5O8n60P+P3mNzNvvmK+85nfd1m0iFGjRjkemzFjBpGR\nkZhMnRtEvvDCC47hHOLa4iMCmDY2hjXbS/lyWwm3Tojv9jkeRg/HipFwdnhPa3srtqZKjp8p2spO\nllPWaKOgrpCCusIurxFkDiTa/+zcts5frXh7ePf0H1FEztPW0UZtSz01LbVn/zlVR01LHTUttdS1\n1NFmbwfgN5N+SaiP5vica/fu3URFRREeHu44949//INFixZd9PpPP/2UL7/8ErPZzBNPPEFcXFxf\nRe131u08TmVdCzelxxIe7OPsOCIivabbAi03N5cjR46wbNkyCgsLWbRoEcuWLetyzeuvv46fn1+v\nhZTeM29KMrl7bXy4oZiM1AjCw69tyIinyZPYgGhiA6K7nD/VfprykzaOn1O0lZ20sa+mgH01BV2u\ntXiHnJ3b5hdBlH8Ekb4RmE2e1/znExlomttaHIVXbcvZwuubfxpON2LHftHnBpj9iQmIxuIVzJCI\nRIK9Avs4vetbvnw58+bNcxzbbDaampqIj7/wC66pU6eSkZHB+PHjWbVqFU899RR//vOfL/v6ISG+\neHiYrjvntf4sd4Yrydp8qo1V2UfwNpv43h1phAQ47wu9/ta2rkJZe4c7ZQX3ytubWbst0LKzs7n5\n5psBSElJob6+nsbGRvz9Nb+oP/D38WT+1BTeXH2A99YV8VhSz+4n42UykxAYR0Jg12+Nm9uaKTtZ\n0WV+W9lJG3nV+8mr3u+4zoCBMB/L2dUkz9x5i/ANP/+tRPo9u91Ow+lGak/Vnld41Tl+bW5rvuhz\njQYjIV5BDApOwuIdgsU7GIt3CCHf/OoV3OXLkG/mo0pXOTk5PPHEE47jdevWkZGRcdFrzx9tcu6w\nyEuprW267ozu9O/uSrOu3FhMXeMp7sxMpK2llcqW1j5Id6H+2LauQFl7hztlBffK2xNZL1fgdVug\nVVVVkZaW5ji2WCxUVlZ2KdAWL15MaWkp6enpPProo1rRz81MHR3N2h2lbNhTxryjtYT49P68Ex8P\nH5KDEkgOSuhyvrH15Jm7bOcUbo02dlXls6sq33Gd0WAkKsCK1dvaZTuAcJ9QTMbr//ZZxBnaO9qp\nPdV1+GGXu2Cn6mjraLvoc80mMxbvEJKC4jsLMK/gM4VYZzEW5BWo/Q2vk81mw8/PD7PZ7Di3Z88e\npk+fftHrn3rqKWbNmsW4cePIzc1l8ODBfRW1X2loOs2nOUfx9/G8oqH4IiLu7qo/idvtXYfGPPLI\nI0yZMoWgoCAefvhhVq9ezaxZsy75/IE4fANcP+/DC8bw+KsbeOZvudw3axjT0+Mwmfr+w1w4ASTR\ndX6b3W6nvqWBYw1lHKs/zrH6M782HKe0oZwd5zzfw+hBTEAEcUHRxAZFER8UTWxQNFa/UKd/OHX1\nvwPnc6e87pK1pbWFFvMJqppqqDxZc+bXaqpO1lDZVENtc/0lhx8GevmTEBRDqF8I4b6hhPtZCPO1\nEO4XSphvCP5mvx7/csxd2rWvVFZWYrFYLjgXGhra5fjll1/mN7/5DQsWLGDx4sV4eHhgMBh46qmn\n+jpyv/Cv7CO0nG7n3puS8fHSwjUi0v8Z7OdXXOd5+eWXCQ8PZ+HChQDcdNNNfPjhhxcd4rh06VKq\nq6t55JFHLvl6PXHr0p1ugYL75P1402E+2nSY1rYOIi2+zJ2SxLhhVowuekc0LMyfgyUlXea3HT9Z\nTvlJG6c7ug5/MRs9ifSznl1N8sxQyRCvvtl8213+DnzDnfK6Sla73U5j68nzhhx+cwes89zJtosP\nXzMajASZAx13vELPG35o8Q7GbDJf9Lm9pafaVUXe1RlofWR3Wavqm1n02maC/Lx45kcZeHo4/4u2\n/tK2rkRZe4c7ZQX3yuv0IY6ZmZm8/PLLLFy4kPz8fKxWq6M4O3HiBP/5n//JkiVLMJvNbNmyhVtv\nvfW6worzzJmcyB1TB/G3j/LYsLuMP32YT1z2EeZNSWb0oFCXG7pqMBgI8Q4mxDuYtNChjvMd9g5q\nWmo7h0ieM8ft+EkbR8/bfNvb5HVmFcmz89ui/SIJNAe43J9XnKu9o526Uw1nC69TdRfM/2rtuPi8\nGE+jJxbvEAaFJeJvDOgyB8ziHUyQOVBDc0Uu4sOvi2lrtzN3SpLTizMRkb7SbYF2ww03kJaWxsKF\nCzEYDCxevJgVK1YQEBDAzJkzycrK4tvf/jZeXl6kpqZednijuL6wYB++N2sYt02M58MNh9mcX84f\n3ttNSnQg87KSSU20dP8iTmY0GAnzCSXMJ5SRYamO8+0d7VSd2Xz7uGOOm40jJ0ooPm/zbV8Pn867\nbf5n57dF+0Xib9Zqpf3V6fbTFxRc5y7CUX+6gQ57x0Wf6+fpS6Sf9Zy5X+fO/wrBz9MXg8HgVt8O\nijhbSWUjm/LKiQn3Y1JapLPjiIj0mSsazP3zn/+8y/GwYcMcv//e977H9773vZ5NJU5nDfHlh3ek\nMjsjng++LmZbQSUvvL2TYfHBzJ+awqCYIGdHvGomo4kIPysRflbGMNJx/uzm2+VnhkvaKGssp6j+\nMIX1xV1eI8DT/5w7bWe3BPD11J48rsxut3Oyremyww8bW09e9LkGDAR7BZEYGH9B4WXxDibEKxhv\nD68+/hOJ9H8r1hVhB+6amoLRqBENIjJwaLatXFZMuD8Pzx/J4fIGVqwvIq+ohmfe3MaolFDmZyUT\nH+H+80vO3Xw7/Zzzp9tbsTVVnNl4++xwyYLaQxTUHuryGsFeQY6hktFn7rxF+kbog3sf6bB3UH+q\ngeqLFF7frH54uv30RZ/rafQgxDuYWP/oLoXXN8VYsFeQhh+K9LGDJXXsPFTF4NggRqeEdv8EEZF+\nRAWaXJHEyED+654xFByrY8X6InYXVrO7sJpxw6zMm5JEVGj/G/pnNnkSFxBDXEBMl/MtbaewNVV0\nmd92qc23Q71DiPKLJDE0mlMt7QDYz/yv8/9n1+j55tzZa878zt7lirPPsV/JNRe+R+e19q7n7V2v\nMXt5cKql9fx0fPNGZ9/jyt77/Nc/+5Quf9JvTnFB6vPOndt+zR3NVDfVXnL4oa+HD1afsC4LbnQu\nxNG5CEeAp7/mG4q4ELvdzvK1hQDcPS1F/32KyICjAk2uypC4YB77X2PJP1zDinVFbN1fwbYDFUxO\ni+TOG5MID+7/Q/28Pbwuuvl2U2sz5U3nLkzSubpkXvU+8qr3OSmtezLQ+YHs3A9mBsdZwOC4An8v\nXxIC4s4bfni2GPP28O7j9CJyPXYVVnOwpJ4xg8IYHBvs7DgiIn1OBZpcNYPBwIikUNISLew4WMX7\nXxexMa+czXttZI2OZs7kREICBt7QPl9PH5KDEkkOSuxyvvH0Sdq9W6ipbQRH4XFOEfJN2eE49835\nswXKOaUJBkOXo87fX3DOwNna5px3Mpx3xnD+K3Ueh4X5U1N98qL5ujzDcEG6bvNdsvi6xm/JtfCG\nSP/R0WHnvXWFGID5U5OdHUdExClUoMk1MxgM3DAknDGDwsjdZ+ODDcWs2VHKhj1lzLghhtkZCQT4\n9u3eTa7I3+xHeGgklR3uU0QEewfQatawIhHpW5v3llNaeZLMEZHEhl+436qIyECgAk2um9FoICMt\nknHDrGzKK2flxmJW5x5j7c7j3Do+jlvGx+Prrb9qIiJyaa1tHby/vhgPk4FvTUlydhwREafRp2bp\nMR4mI1mjo5mUFsHancdZtekwKzce5sttJdyWkcBNN8TiZdZqeCIicqG1O0qpbmhh5rg4woL6/3xm\nEZFLMTo7gPQ/nh4mZo6L4/mfTOauM3MIlq8t5LE/Z/PF1mO0tl18tT0RERmYmk+18dGmw3ibTcyZ\nnODsOCIiTqUCTXqNl9nE7ZMSef4nk7hjciKnWtv55xcHWfRaNut3Hae9Q4WaiIjA6tyjNDa3Mmti\nvOYui8iApwJNep2vtyfzspJ5/ieTuHVCHPUnW/nbJ/t54vUcNu8tp+OcPbJERGRgaTh5mtW5xwj0\n9eSW8XHdP0FEpJ9TgSZ9JtDXzLdnDOb5n0xi2tgYqupbeG3lXn79l1x2FFR22cxYREQGho82HeZU\nazt3ZCbhbdbUeBER/SSUPhcS4MX9tw7ltonxrNxQzKb8cl5esYekqEDmZyWTmhhyzXtiiYiI+yiv\nPsnaHaWEB3szdUy0s+OIiLgEFWjiNOHBPjwwJ5XbMhL44Osith6o5MVlOxkaF8z8qckMjg12dkQR\nEelFSz/dT3uHnXlTkvEwaVCPiAioQBMXEB3mx0PzRnKk/ATvf13E7sJqnv3/tzMyOZT5WckkRAY4\nO6KIiPSwo7YTrNtRQrzVnwmpEc6OIyLiMlSgictIiAzgPxeM5lBJPSvWF7KnqJo9RdWkDw1n7pRk\nYsL8nB1RRER6yIr1RdjtcNe0FIwa1i4i4qACTVzOoNggfnHvWPYeqWXFuiK2Hahke0ElGamRfGtK\nEtZgbWAqIuLODhytZXdhNSNSQhmRZHF2HBERl6ICTVySwWAgLdFCakIIOw9V8f76YrLzy8ndZ2PK\nqCjmTE4kPFxDH0VE3I3dbmf52kIAvnd7qhaFEhE5jwo0cWkGg4Gxg8MZPSiMLfsq+ODrItbuPM6G\nPeXcnpnE9DFRBGpTUxERt7HjYBWFxxu4YUg4wxIsVFaecHYkERGXogJN3ILRYGBiagTjhoWzaU85\nKzcW8+H6Qj7NPszM8bHMmhCPr7ens2OKiMhldHTYeW9dIQYDzM9KdnYcERGXpAJN3IrJaGTK6Ggy\n0iLZXljN258d4ONNR/hqWymzJsZz87hYbXQqIuKiNuaVUVbdxI2joojWwk8iIhelT7Liljw9jMy5\nMZkxyRa+2lbCvzYfYcX6Ir7YeozZkxKZPjYaTw+Ts2OKiMgZrW3tfLihGA+Tkbk3Jjk7joiIy1KB\nJm7Ny9PEbRkJTB0Tw2dbjvLZlmO8/eVBVuce5Y7MRG4cGaXNT0Xkur377rusXLnScZyXl8eIESNo\namrC19cXgMcee4wRI0Y4rmltbeXxxx/n+PHjmEwmnn32WeLi4vo8u6v4clspNQ2nmDUhHkugt7Pj\niIi4LBVo0i/4enswd0oyN6XH8knOUb7cVsI/Pj3Ap5uP8q0pSUwcHoHRqJXCROTaLFiwgAULFgCQ\nm5vLJ598wqFDh3j22WcZMmTIRZ/z8ccfExgYyIsvvsiGDRt48cUXeemll/oytstoamljVfZhfLw8\nmD0pwdlxRERcmm4tSL8S4GvmnumDeO7Hk5hxQwzVDS28/tFeFv8ll20HKrHb7c6OKCJu7tVXX+Wh\nhx7q9rrs7GxmzpwJwOTJk9m+fXtvR3NZn+Ye4WRLG7Mz4vH30YJOIiKXozto0i+FBHhx3y1DmTUh\nnpUbD7Mxr4xX399DQmQA87OSGZFk0d47InLVdu/eTVRUFOHh4QD84Q9/oLa2lpSUFBYtWoS399mh\ne1VVVVgsnZswG41GDAYDp0+fxmy+9NYgISG+ePTA/FlX2ieypqGFz7eWEBLgxcJbh+Pt1fWjhytl\nvRLulFdZe4ey9h53ytubWVWgSb8WFuzDD24fzm0Z8Xy4oZjcfRX8f+/sYkhsEPOnpjAkLtjZEUXE\njSxfvpx58+YBcP/99zN06FDi4+NZvHgxS5cu5YEHHrjkc6/kDn5tbdN1ZwwPD3CpvcXeXH2AU6fb\nuWf6IE40NHNuMlfL2h13yqusvUNZe4875e2JrJcr8DTEUQaEqFA/fvKtEfz638YzOiWUgpJ6nlu6\nnd8t20lxWYOz44mIm8jJyWHs2LEAzJw5k/j4eABmzJhBQUFBl2utViuVlZVA54Ihdrv9snfP+iNb\nbRPrdx0nIsSHKaOinB1HRMQtqECTASU+IoD/Z8FofvXddIYnhJBXXMP//vtWXlmxh5LKRmfHExEX\nZrPZ8PPzw2w2Y7fb+f73v09DQ+cXPDk5OQwePLjL9ZmZmXz66acArFmzhokTJ/Z5Zmd7f30R7R12\n5mUla0VdEZErpCGOMiClxATxi3vHsu9wDSvWF7G9oJIdBZVMTIvgWzcmERHi6+yIIuJiKisrHXPK\nDAYD99xzD9///vfx8fEhIiKC//iP/wDgwQcfZMmSJcyePZtNmzZx7733Yjabee6555wZv88dKT9B\n7r4KEiICGDfM6uw4IiJuQwWaDGjDEy0sSghhV2E1768vYnO+jdy9Fdw4Koo7MxO1V4+IOIwYMYI3\n3njDcTx79mxmz559wXVLliwBcOx9NlAtX1cIwN3TUjBqUSYRkSumAk0GPIPBwJhBYYxKCWXr/go+\n+LqY9buOsymvjGljY7h9UiJBfgNr3oiIyPXYd7iG/OIahieEkJZkcXYcERG3ogJN5AyjwcCE4RGk\nDw0nO8/Gyo3FfLG1hPW7jjNzXByzJsbj5639e0RELsdut3e5eyYiIldHBZrIeUxGIzeOiiIjLYL1\nu47z0abDrMo+wlfbS5k1IY6bx8Xh46X/dERELmbbgUqKy04wbpiVpKhAZ8cREXE7+pQpcgkeJiMz\nboglc2QUa7aX8q/NR3j/62I+31rC7ZMSmD42BrPn9W8oKyLSX7R3dLBifRFGg4H5WcnOjiMi4pa0\n5q1IN7w8TcyaGM/zP5nE3BuTaO/oYNlXh/jla5tZs6OUtvYOZ0cUEXEJG3aXUV7TxJTRUURatBqu\niMi1UIEmcoV8vDy488Yknv/JZG7LiOdkcytvrj7Aotc2s3FPGR0ddmdHFBFxmlOt7Xy4oRizh5E7\nM5OcHUdExG2pQBO5Sv4+niyYNojnfzKJm9JjqWs8xf9ZtY8n/08OW/dX0GFXoSYiA8+X20qoazzN\nzePiCAnwcnYcERG3pTloItcoyN+L78wcwq0T4vho42E27innjx/kER/hz/ysZEYmh2LQ3j8iMgCc\nbGnlX9lH8PP2YHZGvLPjiIi4Nd1BE7lOYUE+/Nvs4Tz1w4lMTI3gqK2Rl97dzbNLt7P/SK2z44mI\n9Lp/bT5C06k2Zmck4KvtSERErovuoIn0kEiLLz++M43ZGQm8v76InYeq+O1bO0hNDOEHd47A4qsP\nLSLS/9SeOMUXW0sICfDipvRYZ8cREXF7KtBEelic1Z9H7h5F0fEG3l9fSP7hWn7+h6+Js/qTkRrB\nhOERhAZ5OzumiEiP+HBDMa1tHXzrxiRtPSIi0gNUoIn0kuToQB5dOJYDR2tZs7OMbfttvLu2kHfX\nFjIkNoiJaZGMGxpOgK/Z2VFFRK5JWfVJNuwuI9LiS+bISGfHERHpF1SgifSyofEh3JgeT/HRGrYd\nqCBnr40DR+soKKnnn58XkJZkYWJqBGMHh+Ft1n+SIuI+3l9fRIfdzvysZExGTWsXEekJ+jQo0kf8\nfTyZOiaGqWNiqD1xitx9NjbvtbG7sJrdhdWYPYyMGRzGxNQIRiaH4mHShx0RcV3FZQ1sPVBJUlQg\n6UPDnR1HRKTfUIEm4gQhAV7cOiGeWyfEU1Z9kpy9NnL22sjdV0Huvgr8vD1IH2olIzWCIXHBGI1a\nrl9EXIfdbmf52kIA7p6Woi1FRER60BUVaM888wy7du3CYDCwaNEiRo0adcE1L774Ijt37uTNN9/s\n8ZAi/VlUqB9zpyTzrRuTOGI7weZ8G7n7bKzfdZz1u44T7G9mwvAIMtIiSIgI0AchEXG6/MM17DtS\ny4gkC8MTQpwdR0SkX+m2QMvNzeXIkSMsW7aMwsJCFi1axLJly7pcc+jQIbZs2YKnp5YRF7lWBoOB\nxMhAEiMDuWf6IAqO1bF5r42t+yv4bMsxPttyjAiLLxOHW8lIiyTS4uvsyCIyAHXY7by3tgiAu6am\nODmNiEj/022Blp2dzc033wxASkoK9fX1NDY24u/v77jmueee42c/+xmvvPJK7yUVGUCMRgPDEkIY\nlhDCd2YOIa+4mpy9NnYerGLlxsOs3HiYhMgAx7L9IQFezo4sIgPE1v0VHLGdYMJwKwmRAc6OIyLS\n73RboFVVVZGWluY4tlgsVFZWOgq0FStWMGHCBGJiYnovpcgA5ulhZOzgcMYODqfldBs7DlaRs9dG\nXlENR8pP8M5XhxgaH8zE1AjSh1rx99GdbBHpHW3tHaxYX4TJaGBeVrKz44iI9EtXvUiI3W53/L6u\nro4VK1bw17/+FZvNdkXPDwnxxcPj+jeyDA93r2/t3CmvsvaOnsoaFxPCndMGU994ik27j7NuRyn5\nRdXsP1rH0s8LSB8WQdbYGCakRuLtde3rAA3Etu0Lyiru7Otdx6mobWb6DTFEhGiYtYhIb+j205vV\naqWqqspxXFFRQXh453K6mzdvpqamhu985zucPn2ao0eP8swzz7Bo0aJLvl5tbdN1hw4PD6Cy8sR1\nv05fcae8yto7eivruMFhjBscRnV9i2PZ/pz8cnLyy/HyNDF2SBgZqRGkJlquatl+tW3vGIhZVeT1\nH6dOt7Ny42HMnkbunJzo7DgiIv1WtwVaZmYmL7/8MgsXLiQ/Px+r1eoY3jhr1ixmzZoFQElJCb/8\n5S8vW5yJSO8IDfLmtowEbstIoLTqm2X7y9mcb2Nzvg1/H0/GDetctn9QbBBGrQQpIlfp863HqD95\nmjmTEwny17xXEZHe0m2BdsMNN5CWlsbChQsxGAwsXryYFStWEBAQwMyZM/sio4hchZgwP+ZnJTNv\nShJFZQ2O/dXW7ihl7Y5SLIFeTBwewcTUCOKs/lq2X0S61djcyic5R/D38WTWhHhnxxER6deuaILK\nz3/+8y7Hw4YNu+Ca2NhY7YEm4kIMBgMp0UGkRAfx7RmD2H+0jpx8G9sKKvgk5yif5BwlKtSXjNTO\nYs2q+SQil/Tuu++ycuVKx3FeXh5vvfUWv/nNbzAajQQGBvLiiy/i4+PjuGbFihX8/ve/Jz6+s6CZ\nPHkyDz74YJ9n7wmrsg/TfKqdb89Iwtf72ue2iohI9/RTVmQAMBmNpCVaSEu08N1bh7C7sIacveXs\nPFTN+18X8/7XxSRFBZ5Ztt+qeUMi51mwYAELFiwAOvcH/eSTT3jqqad4/PHHGTVqFM8//zwrVqzg\nO9/5TpfnzZ49m8cee8wZkXtMdX0LX27rvPs+4wat2Cwi0ttUoIkMMJ4eJtKHhpM+NJzmU21sL6gk\nZ6+N/MM1FJc18PZXBxk9KJyxg0NJHxKOr7eW7Rc516uvvsoLL7yAj4+PY062xWKhrq7Oycl6x4cb\nimlr72Dujcl49sAqzCIicnkq0EQGMB8vDzJHRpE5Mor6k6fZur+CzXvL2Xmwkp0HK3lz9QFGpXSu\nBDkqJRSzpz6cycC2e/duoqKiHKsZAzQ1NfHhhx/y+9///oLrc3NzeeCBB2hra+Oxxx4jNTW1L+Ne\nt9Kqk2zMKyM6zI/JIyKdHUdEZEBQgSYiAAT5mbkpPZab0mNpNxr5dGMRm/NtbC+oZHtBJd5mEzcM\nCScjNYLhiSGYjFe+bL9If7F8+XLmzZvnOG5qauLBBx/kBz/4ASkpKV2uHT16NBaLhWnTprFjxw4e\ne+wxPvroo8u+vqvtFfrax3ux2+Hf7kgjIiKwR17zfO42pNqd8ipr71DW3uNOeXszqwo0EblAZKgf\nt09K5PZJiZRUNHbur7bXxqa8cjbllRPo68n4YRFMTIsgJTpQK0HKgJGTk8MTTzwBQFtbGw899BBz\n5sxh/vz5F1ybkpLiKNrGjh1LTU0N7e3tmEyXLsBcaa/QQ6X1bM4rZ1BMEMlWv17Zw8+d9gYE98qr\nrL1DWXuPO+XtiayXK/BUoInIZcVa/bnb6s9dU5MpLG1g895ytuyv4MvtJXy5vYSwIG8mnlkJMjbc\n39lxRXqNzWbDz88Ps9kMwOuvv86ECRMci4ec7/XXXycqKoo5c+ZQUFCAxWK5bHHmSux2O8vXFgJw\n97QUfQkjItKHVKCJyBUxGAwMig1iUGwQC28azL4jteTstbGtoJJV2UdYlX2E2HC/zmJteARhwT7d\nv6iIG6msrMRisTiOly5dSmxsLNnZ2QBMnDiRn/70pzz44IMsWbKEO+64g1/84he8/fbbtLW18fTT\nTzsr+lXbU1RDwbE6RqWEMiQu2NlxREQGFBVoInLVPExGRiaHMjI5lPtb29lVWM3m/HL2FFXz3roi\n3ltXxKCYICamRjB+mJVAP7OzI4tctxEjRvDGG284jjds2HDR65YsWQJAZGSkW+4P2mG38966QgzA\nXVNTur1eRER6lgo0EbkuZk8T44dZGT/MSlNLK9sOVLJ5r439R2o5VFrPW18cJDUphInDI7hhSDg+\nXvqxI+LKcvbaOFbRyKS0COKsGrYsItLX9ElJRHqMr7cnU0ZHM2V0NHWNp9iyr4LNe23kFdWQV1TD\nP1YfYPSgzmX7RyaH4umhlSBFXElbewfvry/CZDQwd0qys+OIiAxIKtBEpFcE+3sxc3wcM8fHYatt\nIufMSpBb91ewdX8FPl4epA/tXLZ/WHwIRqMWIRBxtrU7Sqmqb+Hm9FjCNY9URMQpVKCJSK+LCPHl\nzswk7picyLFzlu3fsLuMDbvLCPIzM2F450qQSVEBWjFOxAmaT7Xx0abDeJlNzJmc6Ow4IiIDlgo0\nEekzBoOB+IgAN0umtwAAFSlJREFU4iMCuHtaCgeP1ZGzr4It+2x8vvUYn289hjXYx7Fsf3SYn7Mj\niwwYn285xommVu7MTNTCPiIiTqQCTUScwmgwMDQ+hKHxIfyvmweTX1xDzl4b2w9W8tGmw3y06TDx\nEf6OZfstgd7OjizSbzU0neaT3KME+Hpy64R4Z8cRERnQVKCJiNN5mIyMHhTG6EFhnDrdzs5DVeTs\ntbGnqJp31xTy7ppChsQFk5EawbhhVvx9PJ0dWaRf+XjTYU6dbmd+VrJWWhURcTL9FBYRl+JlNjmG\nODY2t7L1QAW5e20cOFpHwbE6ln5eQFqShYzUCMYMDnN2XBG3V1XXzNodpYQFeTNtTIyz44iIDHgq\n0ETEZfn7eDJtTAzTxsRQ09BC7r4Kcvba2F1Yze7CasyeRtKHRTAkJpCRyaEaBilyDT7YUExbu525\nU5K09YWIiAtQgSYibsES6M2sifHMmhhPWfVJx7L92XvKyN5TBkBMmB8jki2MSA5lSGywPmyKdKOk\nopHsvHJiw/3ISI10dhwREUEFmoi4oahQP+ZOSWbulGRaMbBu61HyimvYf6SW1bnHWJ17DLOnkeHx\nIYxIDmVksgVriK+zY4u4nPfWFWIH7pqaor0IRURchAo0EXFr0eH+3DwujpvHxdHa1k7BsXr2FFWz\np6iaXYWd/wBYQ3wYmRTKyBQLQ+ND8PI0OTm5iHMVHKtjV2E1Q2KDGJUS6uw4IiJyhgo0Eek3PD1M\npCVZSEuysPCmwVTVN5NXXENeUQ17D9fw5fYSvtxegofJyNC4IEYkhzIiOZToUF9tji0Dit1uZ/m6\nQgDunjZIf/9FRIC1a79k2rSbur3u6aefZs6cu4iO7p2FlVSgiUi/FRbk41hkpK29g8LSevYU1ZBX\nVE3+4VryD9ey7KtDhAZ6dRZrSaGkJoZomXHp93YdquZQST1jBoUxKDbI2XFERJyurOw4X3yx+ooK\ntF/96ldUVp7otSz6FCIiA4KHyejYGPvuaSnUNZ4iv7iGPUXV5BfXsG7ncdbtPI7JaCAlJoiRyRZG\nJIUSH+GvuwvSr3R02HlvXSEGA9w1NdnZcUREXMLvfvc8+/blM2XKeG655TbKyo7z0kt/5Nlnf0Nl\nZQXNzc384Ac/IjNzCt/97nf56U//izVrvuTkyUaOHj1CaWkJjzzyKJMmZV53FhVoIjIgBft7kTky\nisyRUXR02Ckua2BPUTV5xTUcPNa559p764oI8jMzIqlzZci0JIs2yRa3l51fTmnVSTJHRhIT7u/s\nOCIiF3jnq0Ns2V/Ro685fpiVe2YMuuTj9977XVaseIekpBSOHj3MH//4BrW1NUyYkMFtt82htLSE\nJ598nMzMKV2eV1Fh44UX/sDmzZv48MP3VKCJiPQE45m7ZikxQcydksyJptPkH+6cu5ZXVM3GvHI2\n5pVjAJKiO/dcG5FsISkyUCvfiVtpbWvng6+L8DAZmHuj7p6JiFzM8OFpAAQEBLJvXz4rV67AYDDS\n0FB/wbWjRo0BwGq10tjY2CPvrwJNROQ8Ab5mMlIjyUiNpMNu55itsfPuWlE1h0obKDrewIcbivHz\n9iAtydJZsCVZCPL3cnZ0kctas+M41Q2nuGV8HKFB2thdRFzTPTMGXfZuV2/z9OwcLfP555/S0NDA\nq6++QUNDA//+79+94FqT6eyq0Ha7vUfeXwWaiMhlGA0GEiIDSIgMYM7kRJpa2th3pIY9RZ3z13L3\nVZC7r3MYRnyEv6NYS4kJwsOkjbLFdTSfauPjTYfx8TJx+6QEZ8cREXEpRqOR9vb2Lufq6uqIiorG\naDSybt1XtLa29kkWFWgiIlfB19uD9KFW0odasdvtHK862bkyZHE1BcfqOGprZFX2EbzNJlITLYxI\ntjAyKVR3K8TpPs05SmNzK/OmJBHga3Z2HBERl5KQkMSBA/uJioomODgYgGnTZvD44//F3r153H77\nnVitVv7619d7PYsKNBGRa2QwGIgJ9ycm3J9ZE+M5dbqd/UdrHRtlby+oZHtBJQDRYX5MSIskJTKA\nIXFBeHpoo2zpO/UnT/PZlmME+pm5ZXy8s+OIiLickJAQVqxY1eVcVFQ0f//7247jW265DYDw8AAq\nK0+QnHx2GGZy8iBeeeW1HsmiAk1EpId4mU2MHhTG6EFhANhqm8g7MxRy/5FaPjizMbDZ08iw+BDH\nYiMRIb7OjC0DwEcbiznV2s6C6Sl4mfXlgIiIK1OBJiLSSyJCfIlI9+Wm9Fha29qxNZxmw44S8opr\n2F1Yze7CagCswT6MSO5cyn94fIg+QEuPqqhrZt3O44QHe5M1OtrZcUREpBsq0ERE+oCnh4mxQ63E\nWnwAqK5vIa+4mj1FNew9XMNX20v5anspHiYDQ+KCGZEUyshkC9FhftooW67LB+uLaO+wMy8rWQvX\niIi4ARVoIiJOEBrkzdQxMUwdE0NbeweFpfXkFdewp7CavYdr2Xu4lnfWQEiAFyOTO5fyH55gwddb\nP7ad4d1332XlypWO47y8PN566y1+/etfAzB06FD+53/+p8tzWltbefzxxzl+/Dgmk4lnn32WuLi4\nvozNUdsJNu+1ER/hz4ThEX363iIicm3U04uIOJmHycjQ+BCGxodw19QU6hpPkV/cOXctv7iG9bvK\nWL+rDKPBwKCYQEYkhzIyOZS4CH+MurvWJxYsWMCCBQsAyM3N5ZNPPuHpp59m0aJFjBo1ikcffZR1\n69YxdepUx3M+/vhjAgMDefHFF9mwYQMvvvgiL730Up/mXn5m3uPdU1P0d0VExE2oQBMRcTHB/l5k\njowic2QUHR12issb2FNYTV5xDQdL6ikoqWfF+iIC/cyMSOpcyj8t0aKl0/vIq6++yrPPPst9993H\nqFGjAJg+fTrZ2dldCrTs7Gzmzp0LwOTJk1m0aFGf5tx/pJa8ohqGxQeTlmTp0/cWEZFrpwJNRMSF\nGY0GUqKDSIkOYu6UZBqbW8kvriGvqJo9xTVsyitnU145BiAxKtAxHDIpKhCjUXdMetru3buJiorC\nZDIRGBjoOB8aGkplZWWXa6uqqrBYOgsjo9GIwWDg9OnTmM29X0jb7XbH3bO7pqVoHqOISA+5++47\n+Ne/VnV/4XVQgSYi4kb8fTyZmBrBxNQIOux2jtkaHYuNHCqpp7isgZUbD+Pn7UFakoURSZ1L+Qf7\nezk7er+wfPly5s2bd8F5u93e7XOv5JqQEF88emCPvEJbI0XHG5g0MoqM0bHX/Xq9KTw8wNkRroo7\n5VXW3qGsvccd8prOLLbUm1lVoImIuCmjwUBCZAAJkQHcPimRppY29h3p3Cg7r7ia3H0V5O6rACDO\n6s/I5M6VIVNigrSa3zXKycnhiSeewGAwUFdX5zhvs9mwWq1drrVarVRWVjJs2DBaW1ux2+3d3j2r\nrW267owWix9//SgfgwHmZMRTWXniul+zt3yz2au7cKe8yto7lLX3ODvvD37wHZ555kUiIyMpLy/j\nl798lPBwK83NzbS0tPCzn/2C1NQRtLd3AFx31ssVeCrQRET6CV9vD9KHhpM+NBy73c7x6ibyiqrJ\nK6rmwLE6jlU08q/NR/A2mxiecHaj7LAgH2dHdws2mw0/Pz9HkZWcnMzWrVsZN24cn332Gd/97ne7\nXJ+Zmcmnn37KlClTWLNmDRMnTuyTnF9uPUZZdRNZo6OICvXrk/cUEelJKw59zI6KPT36mmOtI5k/\naM4lH8/Kms7Gjeu56657+PrrdWRlTSclZTBZWdPYtm0LS5f+naef/n97NNOlqEATEemHDAYDMWF+\nxIT5ceuEeE6dbmf/0c5FI/YUV7PjYBU7DlYBEBXq6yjWhsYF49kDQ+z6o8rKSsecMoBFixbx3//9\n33R0dDB69GgmT54MwIMPPsiSJUuYPXs2mzZt4t5778VsNvPcc8/1esbTre28tXo/nh5G7sxM6vX3\nExHpL7KypvPKKy9x1133sGHDOn7605/x9ttv8tZbb9La2oq3t3efZVGBJiIyAHiZTYweFMboQWEA\n2GqbOou1omr2H63lsy3H+GzLMcwenUv+TxsXx5ikEC0ucY4RI0bwxhtvOI4HDRrEP//5zwuuW7Jk\nCYBj77O+9NX2UqrqW5g1MR5LYN99mBAR6UnzB8257N2u3pCcnEJ1dSU2WzknTpzg66/XEhZm5ckn\n/zf79+/llVf6bpsUFWgiIgNQRIgvEem+3JQeS2tbOwUl9WeGQ3YWbXuKqnnuxxlYQ3ydHVWuwp6i\navx9PJmdkeDsKCIibmfSpBt57bU/MmXKVOrqaklJGQzAunVraGtr67McKtBERAY4Tw8TaYmde6l9\newbUNLRg9zAR6uvp7GhylR64fThBwb6YOjqcHUVExO1MnTqdn/zkB/ztb2/R0tLMU08tZs2aL7jr\nrnv44ovPWLVqZZ/kUIEmIiJdWAK9nb6allwbS6A34aF++ncnInINhg9PY926HMfx0qXLHb+/8cap\nANx++534+fnR1NR7P2e1zrKIiIiIiIiLuKI7aM888wy7du3CYDCwaNEiRo0a5XjsnXfeYfny5RiN\nRoYNG8bixYs1qVxEREREROQadHsHLTc3lyNHjrBs2TKefvppnn76acdjzc3NrFq1iqVLl/L2229T\nVFTEjh07ejWwiIiIiIhIf9VtgZadnc3NN98MQEpKCvX19TQ2NgLg4+PD3//+dzw9PWlubqaxsZHw\n8PDeTSwiIiIiItJPdVugVVVVERIS4ji2WCxUVlZ2uea1115j5syZzJo1i7i4uJ5PKSIiIiIiMgBc\n9SqOdrv9gnM/+tGPuP/++/nhD39Ieno66enpl3x+SIgvHh6mq33bC4SHB1z3a/Qld8qrrL3DnbKC\ne+VV1t7hTllFRET6i24LNKvVSlVVleO4oqLCMYyxrq6OgwcPMn78eLy9vcnKymL79u2XLdBqa5uu\nO7S7Lf/sTnmVtXe4U1Zwr7zK2jt6KquKPBERkavT7RDHzMxMVq9eDUB+fj5WqxV/f38A2traePzx\nxzl58iQAe/bsISkpqRfjioiIiIiI9F/d3kG74YYbSEtLY+HChRgMBhYvXsyKFSsICAhg5syZPPzw\nw9x///14eHgwdOhQbrrppr7ILSIiIiIi0u9c0Ry0n//8512Ohw0b5vj9/PnzmT9/fs+mEhERERER\nGYAM9out+iEiIiIiIiJ9rts5aCIiIiIiItI3VKCJiIiIiIi4CBVoIiIiIiIiLkIFmoiIiIiIiItQ\ngSYiIiIiIuIiVKCJiIiIiIi4iCvaB82ZnnnmGXbt2oXBYGDRokWMGjXK8dimTZv43e9+h8lkIisr\ni4cfftiJSS+fdcaMGURGRmIymQB44YUXiIiIcFZUAAoKCnjooYf4/ve/z3333dflMVdr28tldbW2\n/e1vf8u2bdtoa2vjxz/+MbfccovjMVdr18tldaV2bW5u5vHHH6e6uppTp07x0EMPMX36dMfjrtSu\n3WV1pXY9V0tLC3PmzOGhhx7qsrelK7WtdOVO/SO4Vx+p/rH3qI/seeoje5dT+ke7C8vJybH/6Ec/\nstvtdvuhQ4fs99xzT5fHb7vtNvvx48ft7e3t9nvvvdd+8OBBZ8S02+3dZ50+fbq9sbHRGdEu6uTJ\nk/b77rvP/sQTT9jffPPNCx53pbbtLqsrtW12drb93//93+12u91eU1Njnzp1apfHXaldu8vqSu26\natUq+2uvvWa32+32kpIS+y233NLlcVdq1+6yulK7nut3v/udff78+fb33nuvy3lXals5y536R7vd\nvfpI9Y+9R31k71Af2buc0T+69BDH7Oxsbr75ZgBSUlKor6+nsbERgGPHjhEUFERUVBRGo5GpU6eS\nnZ3tklldkdls5vXXX8dqtV7wmKu17eWyuprx48fz+9//HoDAwECam5tpb28HXK9dL5fV1cyePZsf\n/vCHAJSVlXX5Ns3V2vVyWV1VYWEhhw4dYtq0aV3Ou1rbylnu1D+Ce/WR6h97j/rI3qE+svc4q390\n6SGOVVVVpKWlOY4tFguVlZX4+/tTWVmJxWLp8tixY8ecERO4fNZvLF68mNLSUtLT03n00UcxGAzO\niAqAh4cHHh4X/9fvam17uazfcJW2NZlM+Pr6ArB8+XKysrIct+ldrV0vl/UbrtKu31i4cCHl5eX8\n6U9/cpxztXb9xsWyfsPV2vX555/nySef5IMPPuhy3lXbVtyrfwT36iPVP/Ye9ZG9S31kz3NW/+jS\nBdr57Ha7syNcsfOzPvLII0yZMoWgoCAefvhhVq9ezaxZs5yUrn9xxbb94osvWL58OX/5y1+cmuNK\nXCqrK7br22+/zb59+/jFL37BypUrnd4ZXs6lsrpau37wwQeMGTOGuLg4p2WQ6+dO/SOoj+wrrtqu\n6iN7h/rInuXM/tGlhzharVaqqqocxxUVFYSHh1/0MZvN5tRb/JfLCjB37lxCQ0Px8PAgKyuLgoIC\nZ8S8Iq7Wtt1xtbb9+uuv+dOf/sTrr79OQECA47wrtuulsoJrtWteXh5lZWUADB8+nPb2dmpqagDX\na9fLZQXXaleAtWvX8uWXX3LPPffw7rvv8sc//pFNmzYBrte2cpY79Y/Qf/pIV2zby3HFdlUf2fPU\nR/YOZ/aPLl2gZWZmsnr1agDy8/OxWq2O4RCxsbE0NjZSUlJCW1sba9asITMz0yWznjhxggceeIDT\np08DsGXLFgYPHuy0rN1xtba9HFdr2xMnTvDb3/6WP//5zwQHB3d5zNXa9XJZXa1dt27d6vj2sqqq\niqamJkJCQgDXa9fLZXW1dgV46aWXeO+993jnnXdYsGABDz30EJMnTwZcr23lLHfqH6H/9JGu2LaX\n4ortqj6yd6iP7B3O7B8NdhcfF/HCCy+wdetWDAYDixcvZu/evQQEBDBz5ky2bNnCCy+8AMAtt9zC\nAw884LJZ//73v/PBBx/g5eVFamoqTz75pFNvPefl5fH8889TWlqKh4cHERERzJgxg9jYWJdr2+6y\nulLbLlu2jJdffpmkpCTHuYkTJzJ06FCXa9fusrpSu7a0tPCrX/2KsrIyWlpa+OlPf0pdXZ1L/izo\nLqsrtev5Xn75ZWJiYgBcsm2lK3fqH8F9+kj1j71HfWTvUB/Z+/q6f3T5Ak1ERERERGSgcOkhjiIi\nIiIiIgOJCjQREREREREXoQJNRERERETERahAExERERERcREq0ERERERERFyECjQREREREREXoQJN\nRERERETERahAExERERERcRH/F34/2A1nOt+kAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7fe34db87e10>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "xRphYPaVWTzY",
        "colab_type": "code",
        "outputId": "a29f02c4-b004-4756-c06e-2438eb0d306e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "# Test performance\n",
        "trainer.run_test_loop()\n",
        "print(\"Test loss: {0:.2f}\".format(trainer.train_state['test_loss']))\n",
        "print(\"Test Accuracy: {0:.1f}%\".format(trainer.train_state['test_acc']))"
      ],
      "execution_count": 75,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test loss: 0.51\n",
            "Test Accuracy: 82.9%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "z_Zrw9rIWTwU",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Save all results\n",
        "trainer.save_train_state()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "bsWE23n-U-Cy",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Using GloVe embeddings"
      ]
    },
    {
      "metadata": {
        "id": "HH_328MjWoM-",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "We just used some randomly initialized embeddings and we were able to receive decent performance. Keep in mind that this may not always be the case and we may overfit on other datasets with this approach. We're now going to use pretrained GloVe embeddings to initialize our embeddings. We will train our model on the supervised task and assess the performance by first freezing these embeddings (so they don't change during training) and then not freezing them and allowing them to be trained. \n",
        "\n",
        "```python\n",
        "pretrained_embeddings = torch.from_numpy(pretrained_embeddings).float()\n",
        "self.embeddings = nn.Embedding(embedding_dim=embedding_dim, \n",
        "                               num_embeddings=num_embeddings, \n",
        "                               padding_idx=padding_idx, \n",
        "                               _weight=pretrained_embeddings)\n",
        "```"
      ]
    },
    {
      "metadata": {
        "id": "r1EAWPBEKoaS",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "def load_glove_embeddings(embeddings_file):\n",
        "    word_to_idx = {}\n",
        "    embeddings = []\n",
        "\n",
        "    with open(embeddings_file, \"r\") as fp:\n",
        "        for index, line in enumerate(fp):\n",
        "            line = line.split(\" \")\n",
        "            word = line[0]\n",
        "            word_to_idx[word] = index\n",
        "            embedding_i = np.array([float(val) for val in line[1:]])\n",
        "            embeddings.append(embedding_i)\n",
        "\n",
        "    return word_to_idx, np.stack(embeddings)\n",
        "\n",
        "def make_embeddings_matrix(words):\n",
        "    word_to_idx, glove_embeddings = load_glove_embeddings(embeddings_file)\n",
        "    embedding_dim = glove_embeddings.shape[1]\n",
        "    embeddings = np.zeros((len(words), embedding_dim))\n",
        "    for i, word in enumerate(words):\n",
        "        if word in word_to_idx:\n",
        "            embeddings[i, :] = glove_embeddings[word_to_idx[word]]\n",
        "        else:\n",
        "            embedding_i = torch.zeros(1, embedding_dim)\n",
        "            nn.init.xavier_uniform_(embedding_i)\n",
        "            embeddings[i, :] = embedding_i\n",
        "\n",
        "    return embeddings"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "FZBLYjtWKoX5",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "args.use_glove_embeddings = True"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "Gj77_BmoHA3s",
        "colab_type": "code",
        "outputId": "30f47b5b-310e-4bcc-d25c-07723dedb0b6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "# Initialization\n",
        "dataset = NewsDataset.load_dataset_and_make_vectorizer(args.split_data_file, \n",
        "                                                       cutoff=args.cutoff)\n",
        "dataset.save_vectorizer(args.vectorizer_file)\n",
        "vectorizer = dataset.vectorizer\n",
        "\n",
        "# Create embeddings\n",
        "embeddings = None\n",
        "if args.use_glove_embeddings:\n",
        "    embeddings_file = 'glove.6B.{0}d.txt'.format(args.embedding_dim)\n",
        "    words = vectorizer.title_vocab.token_to_idx.keys()\n",
        "    embeddings = make_embeddings_matrix(words=words)\n",
        "    print (\"<Embeddings(words={0}, dim={1})>\".format(\n",
        "        np.shape(embeddings)[0], np.shape(embeddings)[1]))"
      ],
      "execution_count": 79,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Creating from scratch!\n",
            "<Embeddings(words=3407, dim=100)>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "CC4ofLswt-D3",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "d183b8e5-f418-4b8c-f58e-20f06b31982e"
      },
      "cell_type": "code",
      "source": [
        "# Initialize model\n",
        "model = NewsModel(embedding_dim=args.embedding_dim, \n",
        "                  num_embeddings=len(vectorizer.title_vocab), \n",
        "                  num_input_channels=args.embedding_dim, \n",
        "                  num_channels=args.num_filters, hidden_dim=args.hidden_dim, \n",
        "                  num_classes=len(vectorizer.category_vocab), \n",
        "                  dropout_p=args.dropout_p, pretrained_embeddings=embeddings, \n",
        "                  padding_idx=vectorizer.title_vocab.mask_index)\n",
        "print (model.named_modules)"
      ],
      "execution_count": 80,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<bound method Module.named_modules of NewsModel(\n",
            "  (embeddings): Embedding(3407, 100, padding_idx=0)\n",
            "  (conv): ModuleList(\n",
            "    (0): Conv1d(100, 100, kernel_size=(2,), stride=(1,))\n",
            "    (1): Conv1d(100, 100, kernel_size=(3,), stride=(1,))\n",
            "    (2): Conv1d(100, 100, kernel_size=(4,), stride=(1,))\n",
            "  )\n",
            "  (dropout): Dropout(p=0.1)\n",
            "  (fc1): Linear(in_features=300, out_features=100, bias=True)\n",
            "  (fc2): Linear(in_features=100, out_features=4, bias=True)\n",
            ")>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "44gVZbKbHA7R",
        "colab_type": "code",
        "outputId": "05a65ccf-bf15-42dd-84a7-ffe120aca5cf",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "cell_type": "code",
      "source": [
        "# Train\n",
        "trainer = Trainer(dataset=dataset, model=model, \n",
        "                  model_state_file=args.model_state_file, \n",
        "                  save_dir=args.save_dir, device=args.device,\n",
        "                  shuffle=args.shuffle, num_epochs=args.num_epochs, \n",
        "                  batch_size=args.batch_size, learning_rate=args.learning_rate, \n",
        "                  early_stopping_criteria=args.early_stopping_criteria)\n",
        "trainer.run_train_loop()"
      ],
      "execution_count": 81,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[EPOCH]: 00 | [LR]: 0.001 | [TRAIN LOSS]: 0.50 | [TRAIN ACC]: 81.8% | [VAL LOSS]: 0.44 | [VAL ACC]: 84.1%\n",
            "[EPOCH]: 01 | [LR]: 0.001 | [TRAIN LOSS]: 0.41 | [TRAIN ACC]: 85.3% | [VAL LOSS]: 0.45 | [VAL ACC]: 84.2%\n",
            "[EPOCH]: 02 | [LR]: 0.001 | [TRAIN LOSS]: 0.36 | [TRAIN ACC]: 86.7% | [VAL LOSS]: 0.45 | [VAL ACC]: 83.9%\n",
            "[EPOCH]: 03 | [LR]: 0.001 | [TRAIN LOSS]: 0.30 | [TRAIN ACC]: 89.2% | [VAL LOSS]: 0.46 | [VAL ACC]: 83.9%\n",
            "[EPOCH]: 04 | [LR]: 0.001 | [TRAIN LOSS]: 0.26 | [TRAIN ACC]: 90.5% | [VAL LOSS]: 0.48 | [VAL ACC]: 84.0%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "98MqlEQ0sfTs",
        "colab_type": "code",
        "outputId": "ef300406-fa50-41e2-ed77-f558520cc8c3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 335
        }
      },
      "cell_type": "code",
      "source": [
        "# Plot performance\n",
        "trainer.plot_performance()"
      ],
      "execution_count": 82,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA24AAAE+CAYAAAD1QEO5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xd8lfXd//HXWdmDhOzNSAgJhCl7\nyJKwVIZKRaW17a9WqeOu1opaukR7V62i9dbbuy7UFkEQEQQEARVZYYQ9hewJ2Ts55/dHQiQy1SQn\nJ3k/Hw8fyblyXck7x/BI3uf6Xp/LYLPZbIiIiIiIiEibZbR3ABEREREREbkyFTcREREREZE2TsVN\nRERERESkjVNxExERERERaeNU3ERERERERNo4FTcREREREZE2TsVNpBn16NGD7Oxse8cQERFpNbNn\nz+bGG2+0dwyRdk/FTURERER+kOPHj+Pp6UlISAh79+61dxyRdk3FTaQVVFVV8Yc//IGJEycyadIk\nnnnmGerq6gB49913mTRpEomJicyaNYsTJ05ccbuIiEhbsWLFChITE5k6dSofffRR4/aPPvqIiRMn\nMnHiRB555BGqq6svu33Hjh1MmDCh8dgLH7/00ks88cQTzJo1i7feegur1cqf/vQnJk6cyNixY3nk\nkUeoqakB4Ny5c9xzzz2MGzeOadOm8dVXX7F582amTp3aJPOMGTPYsGFDSz81Is3ObO8AIh3B22+/\nTXZ2NqtXr6a2tpY77riDTz75hHHjxvHiiy+yadMmPDw8+PTTT9m8eTPBwcGX3B4dHW3vb0VERASA\nuro6PvvsM+677z5MJhPPPfcc1dXV5Obm8re//Y2PPvqIgIAAfvOb3/DOO++QmJh4ye29e/e+4tfZ\nsmULK1euxNfXl3Xr1pGUlMQnn3yC1Wpl+vTprFmzhptuuonnnnuObt268eqrr3L48GF+9rOf8eWX\nX5KXl8fRo0eJjY0lMzOT1NRURo0a1UrPkkjzUXETaQWbN2/m7rvvxmw2YzabmTZtGlu3bmXy5MkY\nDAaWLVvG1KlTmTRpEgA1NTWX3C4iItJWfPXVV/Tu3RsPDw8ABg0axKZNmygsLKRfv34EBgYC8Nxz\nz2Eymfjwww8vuX337t1X/Dp9+vTB19cXgIkTJzJmzBgsFgsAvXv3Ji0tDagveK+//joAcXFxbNy4\nEScnJyZOnMjq1auJjY1lw4YNjBs3Dicnp+Z/QkRamJZKirSCc+fO4e3t3fjY29ubs2fPYrFYeOut\nt9izZw8TJ07k9ttv59ixY5fdLiIi0lYsX76czZs3M3DgQAYOHMj69etZsWIFBQUFeHl5Ne7n7OyM\n2Wy+7ParufD357lz53j00UeZOHEiiYmJbNy4EZvNBkBhYSGenp6N+54vlFOmTGH16tUAbNiwgcmT\nJ/+4b1zETlTcRFqBn58fhYWFjY8LCwvx8/MD6l8VXLRoEdu2bWPEiBEsWLDgittFRETsraioiJ07\nd7Jjxw6SkpJISkpi165dHDhwAKPRSEFBQeO+paWl5Ofn4+Pjc8ntJpOp8bpvgOLi4st+3X/84x+Y\nzWZWrVrF2rVrGT16dOPHOnXq1OTzp6enU1NTw3XXXUdtbS2bNm3ixIkTDBs2rLmeBpFWpeIm0gqu\nv/56li1bRl1dHeXl5axcuZLRo0dz7Ngx7r//fqqrq3FycqJXr14YDIbLbhcREWkLVq9ezZAhQ5os\nOTSbzYwYMYLq6mr27NlDeno6NpuNBQsWsGzZMkaPHn3J7f7+/uTl5XH27Fnq6upYtWrVZb/u2bNn\niYmJwcnJiaNHj7J3717Ky8sBGDt2LCtWrADg5MmTzJgxg7q6OoxGI5MnT+Yvf/kLY8eObVxmKeJo\ndI2bSDO78847MZlMjY//+te/cuedd5KWlsaUKVMwGAwkJiY2XrcWFhbG1KlTsVgsuLu784c//IGY\nmJhLbhcREWkLPvroI+bOnXvR9gkTJvDKK6/w5z//mblz52Iymejduzc/+9nPcHZ2vuz2mTNncvPN\nNxMSEsJNN93EkSNHLvl17777bh599FGWL1/OwIEDefTRR3n88cdJSEjgkUce4dFHH2Xs2LG4u7vz\n7LPP4uLiAtQvl3zzzTe1TFIcmsF2fmGwiIiIiEg7lJ+fz/Tp09m8eXOTF1dFHImWSoqIiIhIu7Zo\n0SJ+8pOfqLSJQ1NxExEREZF2KT8/n3HjxpGfn8/dd99t7zgiP4qWSoqIiIiIiLRxOuMmIiIiIiLS\nxqm4iYiIiIiItHFt5nYAeXklP/pz+Pi4UVBQ3gxpWp4jZQXHyqusLcORsoJj5e1oWf39PZspTcfQ\nHL8foeP9nLUWZW05jpRXWVuGI2WFlv8d2a7OuJnNjjMpyJGygmPlVdaW4UhZwbHyKqu0Bkf6f6es\nLcORsoJj5VXWluFIWaHl87ar4iYiIiIiItIeqbiJiIiIiIi0cSpuIiIiIiIibZyKm4iIiIiISBun\n4iYiIiIiItLGqbiJiIiIiIi0cSpuIiIiIiIibZyKm4iIsHnzxmva76mnniIzM6OF04iIiMh3qbiJ\niHRwWVmZbNiw7pr2ffzxxwkJCW3hRCIiIvJdZnsHaC5nsos5lllMjxAve0cREXEozz//N44cOcTI\nkddxww2TyMrK5IUXXuHpp/9MXl4uFRUV3H33/2P48JHceeedzJv3X2zatJGyslJSU1PIyEjn/vt/\ny9Chw+39rYiIiLS6jPwyDp0+x/Sx0S36da6puC1cuJDk5GQMBgPz588nISGh8WNjx44lKCgIk8kE\nwLPPPktgYOAVj2kJa3eksvNILndN7MH1/fRqsIjItfrJT+5k+fIP6NKlG6mpZ3jllf+joOAcgwYN\nYdKkqWRkpPPkk79n+PCRTY7Lzc3h2WcXsX3716xc+aGKm4iIdBhWq43kU/ls3J3O4TMFAMRE+RLl\n795iX/OqxW3nzp2kpKSwZMkSTp06xfz581myZEmTfV5//XXc3d2/1zHNbcaorhxLK2Tx+mP4ejmT\n0M2vRb+eiEhL+ODzk+w6mtusn/O62ABuHdv9mvbt2TMeAE9PL44cOcTHHy/HYDBSXFx00b4JCX0B\nCAgIoLS0tPkCi4iItFFllTV8mZzF53vSyS+qBCA2ohPjBoQzsGcg+fkt9/vwqsVt27ZtjB8/HoBu\n3bpRVFREaWkpHh4ezXrMjxXg48YTdw9m/itb+Z+PDvH7Of2JDPJssa8nItIeWSwWAD77bC3FxcX8\n85//R3FxMb/4xZ0X7Xt+pQWAzWZrtYwiIiKtLSOvlI270/n6UDbVNVaczEZG9Qlh3IAwwgPqO47B\nYGjRDFctbvn5+cTHxzc+9vX1JS8vr0kJW7BgARkZGQwYMIDf/va313RMS4iN9OX/TYvnlRUHeGFp\nMk/cNZDO3i4t+jVFRJrTrWO7X/PZseZiNBqpq6trsq2wsJDg4BCMRiNbtnxOTU1Nq2YSERGxN6vV\nxr6T9cshj6TUL4fs7OXC2BGhjEwIwcPV0qp5vvdwku++qnr//fczcuRIvL29ue+++1i37uLJZNfy\nSqyPjxtms+mq+11N4oiuVFlt/N/Kg7y04gB/mzey1Z/Ua+Xv71hnBB0pr7K2DEfKCo6V155ZBwzo\nzVNPHadr1yg8PFzw9/dkxoxp/PrXv+bEiSPMnDmTkJBglix5GwAfH3fc3Z0b9y0ocMfJyexQz7eI\niMjllFbU8OX+TDbtyWhcDtkz0odxA8Lo290Po7Flz6xdzlWLW0BAAPn5+Y2Pc3Nz8ff3b3x88803\nN74/atQojh8/ftVjLqWgoPx7Bb8Uf39P8vJKGNYzgDPpYWzYnc6fX9/GQ7f2wWxqW3c+OJ/VUThS\nXmVtGY6UFRwrr/2zWli6dFXjo7y8EpydvXnjjfcbtw0dOgaAefPmkZdXwuzZP23c18cnmOeff+Wa\nvwcVPBERaYvSc0vZsDud7Yeyqa614mQxcn3fEMYOCCPMv2VXDl6Lq7aZ4cOHN55FO3ToEAEBAY1L\nHktKSvj5z39OdXU1ALt27SI6OvqKx7SW2eOi6Rftx5GUAt769KiuvxARERERkSbqrFZ2H8vlv9/f\nwx/e2MkXyZl4uTtx65juPHffcO5KjG0TpQ2u4Yxb//79iY+PZ/bs2RgMBhYsWMDy5cvx9PRkwoQJ\njBo1ittuuw1nZ2fi4uJITEzEYDBcdExrMxoN/L8b4/nv9/fy9cFs/LxduHlk11bPISIiIiIibUtp\nRQ1fJGeyaU86Z4urAIiLql8O2aeb/ZZDXsk1XeP28MMPN3kcGxvb+P7cuXOZO3fuVY+xB2eLiQdm\nJfDXd5L4eOsZ/LxdGZEQbO9YIiIiIiJiB6k5JWzcnc72wznU1FpxtpgY0y+UsQPCCPVruXuwNYfv\nPZzE0Xi5O/HQrX1YuHg3b689io+XM/FRvvaOJSIiIiIiraDOamXv8Xw27E7neFohAP6dXBjXP4wR\nCcG4ubTNQYbf1e6LG0BwZ3d+MzOBZ/+zl1dWHOCxOQMIC2gba1VFRKT9slqtLFiwgBMnTmCxWPjj\nH/+Im5sbv/vd76irq8Pf35+///3vODk52TuqiEi7U1JeXb8ccm8G5xqWQ8Z38WX8gDB6d+uMsYXv\nu9bcOkRxA4gJ78TPp8Tx2seH+EfDPd58PJ3tHUtERNqxjRs3UlJSwn/+8x9SU1N56qmn8PX15fbb\nb2fSpEk8//zzLFu2jNtvv93eUUVE2o2U7G+XQ9bWWXF2MjG2fyjjBoQR3LltL4e8krY1I7+FDY4L\nZOborhSUVPHC0mQqqmrtHUlExGHMmjWNsrIye8dwKGfOnCEhIQGAiIgIMjMz2bFjB+PGjQNgzJgx\nbNu2zZ4RRUTahdo6KzuP5PD0u7v501u7+OpAFr5ezvxkfDTP3TucO27o4dClDTrQGbfzJg+JJL+o\nki37MvmflQe5f2ZCm7vHm4iItA8xMTG8/fbbzJ07l5SUFNLS0qioqGhcGtm5c2fy8vLsnFJExHEV\nl1ezZV8mm/dmUFBSvxyyV1dfxg8Ip1dXX4dbDnklHa64GQwG7rghhoKSKvafOsu7648zN7EHhnb0\nP1VE5Pu4++45LFz4HEFBQWRnZ/HYY7/F3z+AiooKKisreeihR4iL62XvmA5p9OjR7Nmzhzlz5tCj\nRw+6du3K8ePHGz9+LfcY9fFxw2w2NUseR7r5ubK2DEfKCo6VV1lbxuWynkwrZNVX3/DF3gxq66y4\nOpuZOqILU4Z3ISzAft9fSz63Ha64AZiMRu65KZ5n3tvDF8mZ+HdyYcrQKHvHEhGxi1GjxrB16xfM\nnHkrX365hVGjxtCtWzSjRl3P7t27eO+9t3nqqb/bO6bDeuihhxrfHz9+PIGBgVRWVuLi4kJOTg4B\nAQFXPL6goLxZcvj7e5KXV9Isn6ulKWvLcKSs4Fh5lbVlfDdrbZ2V3cfy2Lg7nZMZRQAE+roxrn8o\nw3sH4+pcX23s9f01x3N7peLXIYsbgIuTmQdm9eGpxUl8uOUbOnu5MCQ+yN6xRKSDW37yE/bmHmjW\nz9kvoDczuk+97MdHjRrDyy+/wMyZt/LVV1uYN+8h/vOfxfz734upqanBxcWlWfN0JEePHuXtt9/m\n6aef5osvviAuLg5vb2/WrVvHTTfdxPr16xk5cqS9Y4qItGlFZdVs2ZfB5r0ZFJZWA5DQrTPjBoQR\n36V9LYe8kg5b3AB8PJ158JY+PP3ubt5YcwQfT2d6RPjYO5aISKvq2rUbZ8/mkZOTTUlJCV9+uRk/\nvwCefPIvHD16mJdffsHeER1WTEwMNpuNWbNm4ezszLPPPovJZOLRRx9lyZIlhISEcPPNN9s7pohI\nm3Q8tYBlG46z62gOtXU2XJ1NjB8Yxrj+YQT6utk7Xqvr0MUNIMzfg/um9+YfHyTz8vIDzL9zgMNP\nnBERxzWj+9Qrnh1rKUOHjuB///cVRo4cTWFhAd26RQOwZcsmams1gfeHMhqNPPPMMxdtf/PNN+2Q\nRkSk7auts5J0NJeNu9M5lVkMQHBnN8b2D2NYr6DG5ZAdUcf9zi8QF+XLTyfF8q/VR/jHB8k8ftdA\nvN11M1QR6ThGjx7DPffczVtv/ZvKygr++tcFbNq0gZkzb2XDhvWsXv2xvSOKiEg7VlRaxeaG6ZBF\nZdUYgOviAhnVO5i4KB8NEkTFrdHw3sHkF1Wy8qvTLFqWzO9+0h9np+aZ4iUi0tb17BnPli07Gh+/\n996yxvdHjBgNwJQpN+Lu7k55uWNc1C4iIm3fqcwiNu5OZ9eRXOqsNlydzdxwXThj+4cSHxPoMINU\nWoOK2wVuHB5FflEFWw9k89rHh5g3ozdGo9q9iIiIiEhzqamtXw65YXc6p7O+XQ45fkAYQ3sF4eKk\ninIpelYuYDAYmJsYy7niKvadzOffG09w+/honZoVEREREfmRCkur2Lw3g837MiluWA7Zt7sf4weG\n0TNSyyGvRsXtO8wmI/dN783T7+1m4+50/L1duGFQhL1jiYiIiIg4HJvNxqnMYjbuTifpaP1ySDdn\nMxMHhTOmfxgBnVztHdFhqLhdgpuLmYdu6cNf3kliyecn8fVyYWDslW+QKiIiIiIi9Wpqrew8ksPG\n3emcya6/Ti3Uz51xA8MYGhekWRI/gIrbZfh6ufDgrD488/4eXv/kMJ08neke6m3vWCIircZms3G2\nsoD00kzSSzJIL82iX1hPBvsOtnc0ERFpowpKqti0N4Mv9mVQXF6DwQD9ov0YPzCc2IhOWg75I6i4\nXUFkkCe/vqkXi5btZ9Gy/Tx+1wACfTrezf5EpP2rs9aRVZZTX9JKM0kvqX9bUVvZZL/OHt4M9rVT\nSBERaZNsNhunMorZsDuN3cfyqLPacHcxkzg4grH9QvHTcshmoeJ2FQndOnPHxBjeWXuMFz5IZv6d\nA/B00z3eRMRxVdRWklGa1VjO0ksyyCrLodZW17iPAQMBbn7E+fYgzDOEcI9QQj2D6RYaotHMIiIC\nQE1tHTsO198sOyWn/ndDmL874waEMSQ+CGeLlkM2JxW3a3B931DyCytZsz2Fl5Yf4JHZfbGY9YMo\nIm2bzWajqLq4oaBlNSx3zCSv4myT/SxGM6EeIYR5BhPmEUqYZwihHsE4m/QilYiIXOxccSWb9maw\nZV8mpRX1yyEHxPgzfmAYMeFaDtlSVNyu0YzRXckvqmDnkVxe/+QI99wUj1E/lCLSRlhtVvLK80m7\nYJljekkmJTWlTfZzN7vRw6c7YR4hhHmGEOYRQqCbPyajXowSEZHLs9lsnEgvYsPudPYcy8Nqq18O\nOWlIBGP6heLnreWQLU3F7RoZDQZ+PiWOwpIqko7msszbhVvHdLd3LBHpgKrrasgqyyatYWBIekkm\nGaWZVFtrmuzX2cWHPt7xjQUt3DOUTs7eeiVURESuWXVNHTsO10+HTM2tfzEwPMCjfjlkXCBOWg7Z\nalTcvgeL2ci8mQksXLybtTtS8fd2YUz/MHvHEpF2rLSmrMkZtPTSTHLK87DarI37GA1Ggt0Dm5xF\nC/MIxs2iYUoiIvLDnC2qXw75RXL9ckijwcDAHv6MHxhOdJheBLQHFbfvycPVwoO39uGpd5J497Pj\n+Hi50Le7n71jiYiDs9lsnKssuGipY0FVYZP9nE1ORHlFEH6+oHmGEOwWiMVksVNyERFpL2w2G8fT\nCuuXQx7Pw2ar/9t3ytBIxvQLxdfLxd4ROzQVtx8goJMrD8zqw3+/v4dXVx7k93P6ExXkZe9YIuIg\n6qx1ZJfnkl6SSVppBrkHc/nmXBoVtRVN9vN28iSucw/CGwaGhHkE4+faGaPBaKfkIiLSHlU1LIfc\nkJROel79csiIQA/GDwhncFyAhvK1ESpuP1DXEC9+dWM8Ly8/wItL6+/xposyReS7Kmsr669Du+BM\nWlZp9kWj9/3dOhPnG/PtckfPELycPO2YXERE2rv8ogo27alfDllWWYvRYOC62ADGDwyje6iWQ7Y1\nKm4/Qr8Yf2aPj+bfG07wwtL9zL+jP24uWq4k0lEVVRWTXppJ2gX3R/vu6H2z0UyIR3CTpY59IqMp\nKay5zGcVERFpPjabjaMpBWzYnc7eE/XLIT3dLEwdFsWYfqH4eDrbO6JchorbjzRhYDj5hZV8lpTG\ny8sP8F+39cVs0jImkfbs/Oj9JiWtNJOS6qaj993MrsT4dCfMI5hwz9DLjt53sbhQgoqbiIi0rKSj\nuazevouU7PqbZUcGeTJ+QBiDemo5pCNQcWsGt43tztniSvYcz+PNNUf5xdSeOrUs0k7U1NWQWZbd\nuMwxrSSTjLIsquuqm+zn6+JDH794QhtH74fg46ybkIqIiP1ZrTY+/OIUn25PxWQ0MKhnAOMHhtMt\nxEu/pxyIilszMBoN/HJaHH//9162HcrGz9uF6aO62juWiHxPZTXljQND0kuyyCjNJLs896LR+0Fu\nAYR5hhB+wfh9jd4XEZG2qKKqlv/9+BDJp84S6OPKgl8OxUWLwxySilszcbaYuH9mAk8tTmLV12fw\n83ZhZJ8Qe8cSkUs4P3r//MCQtKuM3q+/Fi2YcI9Qgt01el9ERBxDbmEFLy3bT0Z+GfFRPtxzcy/C\nAz3JyyuxdzT5AVTcmpGXuxMP3dqXp95J4p11x/D1ciG+i6+9Y4l0aBeO3q9f6phBemnWRaP3vZw8\nifPt0XgGLdwzRKP3RUTEYR1NKeCVjw5SWlHD+IFh3Da2Oyajfqc5MhW3Zhbk68ZvZibw7H/28c8V\nB3jsjgGEB3jYO5ZIh1BWU05maTaZZdnkn87lZH4KmWU51FprG/c5P3q/p290w5m0+qEh3s4avS8i\nIu3D5r0ZvPfZcQB+OimWUVoF1i6ouLWAmPBO/GJqT15deYgXlibzxF0DNVpVpBlV1VWTXZbTWNIy\nS7PJKsumqLrp0g+zwUSIRxBhDTewDvcMIcQ9CBezi52Si4iItJzaOiv/2XiCz/dk4OFq4b7pvegR\n4WPvWNJMVNxayKCegZwtqmTp5lO8sDSZ38/pj6uznm6R76POWkdOeR6ZZdlklWaTWZZDZlk2ZyvO\nYcPWZF8f507Ed44lxD2IYPdAEiKicapyv2j0voiISHtUWlHD/3x0kCMpBYT5u3P/zAT8OrnaO5Y0\nIzWJFpQ4OIK8oko2783gfz46yP2zEnSPN5FLsNqsnKssaDiDlkNmaRZZZTnklOdRZ6trsq+HxZ3o\nTl0J9ggixD2QEI/6ouZqbvrLyb+TLr4WEZGOITO/jEUf7ie3oIJ+0X78YmqcThi0Q/o/2oIMBgNz\nJkRzrriS/afO8u76Y8xNjNX9MqTDstlsFFeXklmW1eQMWlZZzkX3RXMyORHuGdpQzoIJbihpnhYP\n/RsSERFpsP/UWV77+CAVVXVMGRrJ9FFdMer3ZLuk4tbCTEYj99wUz9/e28sXyVn4ebsydViUvWOJ\ntLjymgqyGorZ+WvQMsuyKaspb7KfyWAiyD2gvpi5BxHiEUSIexA+Lp000VFEROQybDYb63amsXTz\nScwmI/9vWhxD4oPsHUtakIpbK3BxMvPALQk89U4Sy7/4Bj9vF6Zdrwl20j5U19WQXZ5DVmlDSWso\naoVVRU32M2DAz9WX7p26EuIeSHBDSQtw9dN1aCIiIt9DTa2Vd9YdZeuBbLw9nLh/ZgJdgr3sHUta\nmIpbK+nk4cyDt/Rh4bt7eGPNEaLCfQjy0qRJcRx11jqyy3IvOoOWV372okEhnZy96ekb0+QMWpB7\nAE4mJzulFxERaR+Kyqr55/IDnMwookuwJ/NmJGh6eQeh4taKQv09mDe9F89/kMxTb+7ksTn9CfFz\nt3cskSZsNhvnKgsbi1lmaQ6ZZVnklOc1uR8agJvZlW6dohomOZ4vaYG4WdzslF5ERKT9Ssku4aXl\n+zlXXMXguEB+NikWJ4tWrXQUKm6trGeULz+dFMu/Vh/hhaXJPH7XQLzddRZC7KOkupSssmwyzp9B\nK80hqyybyrqqJvtZjBYivUPxd/ZvPIMW7BGIt5OXBoWIiIi0gqSjufzf6sNU11iZObork4dE6ndw\nB3NNxW3hwoUkJydjMBiYP38+CQkJF+3z3HPPsW/fPhYvXsyOHTt44IEHiI6OBiAmJoYnn3yyeZM7\nsOG9g6motfH+uqO8uDSZR2/vj7OTXi2RllNZW9k4KCSrNIeMhvuildSUNtnPaDAS6OZ/wRm0QELc\ng+ns6kNggLfG64uIiLQym83Gqq1n+Oir0zhbTPxmRm/6xfjbO5bYwVWL286dO0lJSWHJkiWcOnWK\n+fPns2TJkib7nDx5kl27dmGxWBq3DRo0iEWLFjV/4nZi9oQYUjIL2Xogm9c+PsS8Gb0xGvWqifw4\nNdZacsvzyGi4D9r5a9HOVhZctG9nF196e/ckxD24fliIRxABbv5YjDoRLyIi0hZU1dTxr9VHSDqa\ni5+3C/fPTCAswMPescROrvoX2rZt2xg/fjwA3bp1o6ioiNLSUjw8vv2heeaZZ3jooYd4+eWXWy5p\nO2MwGJibGEtBSRX7Tubz7w0nuH1CtE55yzWx2qzkV5wlsyyHrNLsxjNouRX5WG3WJvt6OXkS6xNN\ncMPZsxCPQILcAnEx60JmERGRtupccSWLPtxPak4pMWHe3DujN15uurymI7tqccvPzyc+Pr7xsa+v\nL3l5eY3Fbfny5QwaNIjQ0NAmx508eZJ77rmHoqIi5s2bx/Dhw5s5uuMzm4zce3NvnnlvNxv3pOPX\nyYWJgyLsHUvaEJvNRlF18QXXoNW/zSrLpcZa02RfF5MLUV7hDdef1V+HFuIehIeTBuCIiIg4klMZ\nRby0/ADFZdWM6hPMHTf0wGzSvU07uu+9Jspm+3bsd2FhIcuXL+fNN98kJyencXtUVBTz5s1j0qRJ\npKWlcdddd7F+/XqcnC7/KoGPjxtm84+/zsvf33Huj3Y+659/NZyHF33BB5tO0iXch+EJIXZOdmmO\n+Nw6gvNZS6vKSC3KJK0ok9SiDNIa3i+rqWiyv8VoJswrmHDvEMK9Q4joVP+2s6tPi5+xdaTnFRwr\nr7KKiAjA1gNZvL32KHVWGz8ZH834AWFakSXANRS3gIAA8vPzGx/n5ubi719/QeT27ds5d+4cc+bM\nobq6mtTUVBYuXMj8+fOZPHm1AjMiAAAgAElEQVQyABEREfj5+ZGTk0N4ePhlv05BQfmP/V7w9/d0\nmOEJ3816/8zePP3eHp57bzdGq5Xuod52THcxR35ufyybzYbVZqXOVlf/n9VKra2WOuuF2xre2qzU\nWWvr3zZsr7XVYbXWf+y7x9UYKzmVn0ZmaRZF1U0zGzAQ4OZHTKfuF5xBC8TPtfNFN6y2lUF+WdNB\nI83NkX4GwLHydrSsKn4iIhezWm0s23KKtTtScXM28+ubexHfxdfesaQNuWpxGz58OC+99BKzZ8/m\n0KFDBAQENC6TTExMJDExEYD09HQee+wx5s+fz8cff0xeXh4///nPycvL4+zZswQGBrbsd+LgIgI9\nue/mXrywdD+Llu3n8bsGEOjTPu6FVV96Li40376t31bbsM16wfv1xzUtRxcWobom+9ZhtVqxpBgp\nLa/8Tqn6ztf9ztewNhSsyxWwlubj3In4zrGNN6wOdg8iyM0fi8ly9YNFRETEoVVU1fLax4fYf+os\ngb5uPDArgSDf9vF3oDSfqxa3/v37Ex8fz+zZszEYDCxYsIDly5fj6enJhAkTLnnM2LFjefjhh9m4\ncSM1NTX88Y9/vOIySanXq2tn7krswVufHuUfHyTz+J0D8LzMRag2m62hfNQ2Fo7z79daaxvO9NTW\nl5ML3r/8Pt9ury803+5fZ6vDaDFQUVl11QJWa6vFesEZpVprHTZsl/we7MloMGIymDAbTZgMJkwG\nI0aDCSeTBZPBBVPjdhMmo7Hh7QXbDEbMRnP9cUYT5sZ96z9mMpi/Pa7JscZvv6bRRKi/H641nria\nXe39lIiIiIgd5BaUs+jDA2Tml9Griy/33BSPm4teuJWLXdM1bg8//HCTx7GxsRftExYWxuLFiwHw\n8PDg1VdfbYZ49nN+edyFZaa+xDS8b6u9dFm6oEjVXG4fWx2WMwZKyyubFKrzRSxwUBmFZRU88eXn\n+HhZGorRxZ/DXgwYGguK2WDGeEFBcTO5fluELio0FxYb00XlyGz4ThG6oOw03bfp9gv3NRpMBHT2\noqiw4hJfo35fo8HYZtaK+/s5zhI5ERERaV5HUgp4ZcUByiprmTAwnFvHdsNk1BASubR2c8OmtJIM\nNmYfo6i0/KICddmzTBeVsqbH2etMkclgwuxmoKbOQEGZGU9XZ5xNzrgZTZiNZswGU8PZnobHDeXk\n/PtNPnZ+X6MJs6FhX6MZi6H+rfk72y+9v4kg/04UnC3HZKwvPm2Zv7cnztUqQyIiItJ2bdqTzvsb\nTgDws0mxjOzTNofTSdvRborbhtQtJOXsu6Z9v10m921RsZgsuBpdv1OKLl9+vrtPk+JkaNi34ViT\n0YzFaMJ0wbH1Z4WqLtq/fsmekZpaK88v2cextEKGDQrntrHRLfwMXpmrxYVSU83VdxQRERGRy6qt\ns/LvjSfYtCcDTzcL903vTUx4J3vHEgfQborbT3rM4Mb4cZQUV2EyXFyUzBc8bgtnjPy9PLFUXf6s\nkMVsZN7M3ixcvJt1O9Pw83Zl3ICwVkwoIiIiIs2ptKKGV1Yc4GhqIWH+Htw/qzd+3rrOXa5Nuylu\nLmYXwv39yaP9LJFzd7Hw4C19eOqdJN7fcJzOXi70jfazdywRERER+Z4y8st4adl+cgsr6Bftxy+n\nxeHi1G7+FJdWYP9TT3JF/p1ceeCWPljMRl79+CCns4rtHUlEREREvof9p/J56p0kcgsrmDosivtm\n9FZpk+9Nxc0BdAn24lc3xlNTa+XFZfvJL6ywdyQRERERuQqbzcbaHam8uHQ/dVYbv7oxnhmjumJs\nI9OtxbGouDmIftH+3D4+huKyav6xNJmySg0KEREREWmramrr+NfqI3yw6STeHk78fk5/BscF2juW\nODCdo3Ug4waEkVdYwfpdabz84QH+67a+WMzq3iIibVVZWRmPPvooRUVF1NTUcN9991FeXs4bb7yB\nxWIhMDCQp59+GicnJ3tHFZFmVFRaxcvLD3Aqs5guwZ7Mm5GAj6ezvWOJg1NxczC3ju3O2eJKdh/L\n481Pj/DLqXFt5mbSIiLS1IoVK+jSpQu//e1vycnJYe7cuZSVlbFmzRo8PT158skn+eyzz5gyZYq9\no4pIM0nJLmHRh/spKKliSFwgP50Ui5PFZO9Y0g6ouDkYo8HAL6fGUVi6l+2HcvDzdmHGqG72jiUi\nIpfg4+PDsWPHACguLsbHxweLxUJxcTGenp6N20Skfdh1NJd/fXKYmlorM0d3ZfKQSL3ALs1Gxc0B\nOVlM/GZmAgvf2c0nX6fg5+3KqD4h9o4lIiLfMWXKFJYvX86ECRMoLi7mtddeo6qqiunTp+Pp6Ulc\nXBzDhg2zd0wR+ZGsVhsfffkNH289g7OTiXkze9Mv2t/esaSdUXFzUF5uTjx0ax+eWrybd9Yew9fT\nmV5dO9s7loiIXGDlypWEhITwr3/9i6NHj/LYY49RW1vLsmXLCA8P58EHH2Tjxo2MGzfusp/Dx8cN\ns7l5lln5+3s2y+dpDcraMhwpKzhG3sqqWv62eBdf788iwNeNJ+8eTFSwl71jXZEjPK/nOVJWaNm8\nKm4OLNDXjftnJvDf/97LPz86yGNz+hMR6Fg/3CIi7dmePXsYMWIEALGxsZw6dYrIyEgiIiIAGDp0\nKAcPHrxicSsoKG+WLP7+nuTllTTL52ppytoyHCkrOEbes0WVvPThflJzS4kJ78S903vhbja06dyO\n8Lye50hZoXnyXqn4aSShg+se5s0vp8VRVV3Hi8v2c6640t6RRESkQWRkJMnJyQBkZGQQFBREUVER\n586dA+DAgQNERkbaM6KI/EAn04v4y9u7SM0tZeKQSB6e3RcvN02IlZajM27twHWxAZwd050PNp3k\nhaX7eeyO/rg663+tiIi93XbbbcyfP5877riD2tpa/vSnP1FWVsY999yDk5MTYWFhmigp4oC2Hsji\n7bVHsVrh9vHRzE7sSX5+qb1jSTunv+7biYmDwskrqmDTngxeWXGAB27pg9mkE6oiIvbk7u7Oiy++\neNH28ePH2yGNiPxYVquNpZtPsm5nGm7OZn59cy/iu/hqcqS0Cv1l304YDAZuHx9Nn26dOXSmgHfW\nHcNms9k7loiIiEi7UF5Zy6IP97NuZxpBvm48MXcg8V187R1LOhAVt3bEZDRyz029iAzy5Kv9WXzy\n9Rl7RxIRERFxeDkF5Ty1OIn9p87Sq4svT9w1gCBfN3vHkg5Gxa2dcXYy8eCsBDp7ubDiy9NsO5ht\n70giIiIiDuvImXP89e0kss6Wc8N14TxwSwJuLhZ7x5IOSMWtHfL2cObBW/vg6mzmjTVHOJJSYO9I\nIiIiIg7n8z3pPLckmcrqOn42KZbZ46IxGfXns9iHfvLaqVA/d+bN6A3Ay8sPkJFfZudEIiIiIo6h\nts7KO+uO8e7647i7mnnkJ/0Y2SfE3rGkg1Nxa8d6Rvpw9+SeVFTV8sIHyRSVVtk7koiIiEibVlpR\nw/NL9rF5bwbhAR48OXcgMeGd7B1LRMWtvRvaK4jpI7twtriSF5btp7K61t6RRERERNqkjLxS/vL2\nLo6mFtI/xp/H7uiPn7ervWOJACpuHcLUYVGMTAgmJbuE11Yeos5qtXckERERkTZl38l8nlq8m7zC\nSqYNi+Le6b1wcdItj6XtUHHrAAwGA3dO7EF8F1+ST53l/Q0ndI83EREREcBms/Hp9hReWrafOquN\ne26KZ/qorhh1U21pY1TcOgizyci9N/cizN+DTXsyWLczzd6RREREROyqpraO//vkCEs3n6KTpzOP\n3dGfQT0D7R1L5JJU3DoQV2czD96SgI+nMx9sOsmuo7n2jiQiIiJiF4WlVfzt/b1sO5RNl2Avnpw7\nkKggL3vHErksFbcOxtfLhQdmJeDiZOL1VYc5kV5o70giIiIirepMdjF/eTuJbzKLGRofyO/n9KOT\nh7O9Y4lckYpbBxQR6Mm9N/fCarXx0ocHyDlXbu9IIiIiIq1i55Ecnnl3D4UlVcy6vhu/mBqHxWyy\ndyyRq1Jx66B6de3MXYk9KK2o4R8fJFNcXm3vSCIiIiItxmqzseKLb3h15SEMRgO/mZnA5CGRGDSE\nRByEilsHNqpPCFOHRZJbWMFLy/ZTXVNn70giIiIiza6quo7/WXGQVV+fwc/bhcfvHEDfaD97xxL5\nXlTcOrjpI7syJD6QU5nFvP7JYay6TYCIiIi0I/lFFSx8dze7j+fRI7wTT84dSJi/h71jiXxvuqtg\nB2cwGPjZpJ4UllSx+1geH3x+ktnjou0dS0RERORHO5FeyD+XH6C4vIbr+4Zw+4QYzCadtxDHpJ9c\nwWI2ct+M3gR3dmP9rjQ27k63dyQRERGRH+XL/Zn89/t7Ka2oZc6EGO6c2EOlTRyafnoFAHcXCw/d\n0gcvdyfe33CcvSfy7B1JRERE5HuzWm38Z+MJ3lxzFBcnE/91Wx/GDQjTEBJxeCpu0sivkysPzErA\nYjby2spDnM4qtnckERERkWtWXlnLC8uSWb8rjeDObjxx10DionztHUukWai4SRNdgr2458Ze1NRZ\neXFpMnmFFfaOJCIiInJVOefKeWpxEge/OUevrr48fudAAn3d7B1LpNmouMlF+kb7cfv4GIrLa3hh\naTJllTX2jiQiIiJyWYfOnOOv7ySRdbaciYPCeXBWH9xcNINP2hcVN7mkcQPCSBwUQdbZcl768ABV\nusebiIiItDE2m42Nu9P5x5JkqmrquHtyT24bG43RqOvZpP3RSxFyWbPGdCO/uJKko7n86ukNTLwu\nnFF9QnCymOwdTURERDq42jor7312nC37MvFyszBvRgLdw7ztHUukxai4yWUZDQZ+ObUnft4ubNqb\nwfsbTrB6WwqJgyO4vm8ozk4qcCIiItL6SsqreWXFQY6lFRIe4MH9MxPo7O1i71giLUrFTa7IYjZx\n65ju3DE5jn+vPcKG3eks+fwka7anMHFQBGP6heLqrB8jERERaR3peaUsWraf/KJKBvTw5xdT4vRi\nsnQI1/QX98KFC0lOTsZgMDB//nwSEhIu2ue5555j3759LF68+JqPEcfh7eHMzNHdmDgogg1JaXyW\nlM6yzaf4dHsKNwyKYFz/MF0ELCIiIi1q74k8/nfVYaqq67hxeBQ3juiCUfdnkw7iqn9p79y5k5SU\nFJYsWcKpU6eYP38+S5YsabLPyZMn2bVrFxaL5ZqPEcfk4Wrh5pFdueG6cDbsTuezXWms+OIb1u1I\nZfzAMCZcF467i8XeMUVERKQdsdlsrNmewvIt32AxG/n1zb24LjbA3rFEWtVVp0pu27aN8ePHA9Ct\nWzeKioooLS1tss8zzzzDQw899L2OEcfm5mLhxuFd+O9fD2Pm6K4YjQY+3nqGR175mg+3nKKkvNre\nEUVERKQdqKmt4/VPDvPhlm/o5OnMY3cMUGmTDumqxS0/Px8fH5/Gx76+vuTl5TU+Xr58OYMGDSI0\nNPSaj5H2w9XZzJShUfz918O4dUx3nCwmVm9L4Xf/s42lm05SXKYCJyIiIj9MYWkVz7y3l+2HcugW\n4sUf5g4kMsjT3rFE7OJ7X5Rks9ka3y8sLGT58uW8+eab5OTkXNMxl+Pj44bZ/OMvLPX3d5x/zI6U\nFa6e987QTtxyQw/Wb0/hw00n+XRHKhv3ZDBpaBQzxnTH16v1pj050nOrrC3HkfIqq4hIUyfSCvjL\n20kUlFQxND6In07qgaUZ/lYUcVRXLW4BAQHk5+c3Ps7NzcXf3x+A7du3c+7cOebMmUN1dTWpqaks\nXLjwisdcTkFB+Q/9Hhr5+3uSl1fyoz9Pa3CkrPD98g7tGcDA6M58uT+LNdtTWPnFKVZvPc3oPiFM\nGhLR4gXOkZ5bZW05jpS3o2VV8RORq9l5JIc3Vh+hptbKLWO6kTgoAoOGkEgHd9WlksOHD2fdunUA\nHDp0iICAADw8PABITExkzZo1fPDBB7z88svEx8czf/78Kx4jHYPFbGJs/zCe+dVQ5ib2oJOHExv3\npPP717bxzrpj5BdV2DuiiIiItCE2m41Dp8/x93/v5dWVhzCZjPxmVgKTBkeqtIlwDWfc+vfvT3x8\nPLNnz8ZgMLBgwQKWL1+Op6cnEyZMuOZjpGMym4yM7hvK8N7BbD+UwyfbzrB5bwZfJmcyrFcQU4ZG\nEuDjZu+YIiIiYidWq42kY7l8uj2VlJz6M/pxUT7cd0tfXE0qbCLnXdM1bg8//HCTx7GxsRftExYW\n1ngPt0sdIx2b2WRkREIwQ3sFsvNwLp9sO8OX+7PYeiCbIfGBTB0WRZCvCpyIiEhHUVNbx9YD2azd\nmUpuQQUGAwyMDWDykAiigrwcahm5SGvQHZOlVZmMRob2CmJwXCBJx3JZtfUMXx/MZtuhbAb1rC9w\noX7u9o4pIiIiLaS8spZNe9P5LCmd4rJqzCYD1/cNYeLgCAK1CkfkslTcxC6MRgODegYyMDaAvcfz\n+HjrGXYczmHn4RwGxAYwbVgU4QG6LlJERKS9KCyt4rNdaWzel0FFVR2uziYmDYlgwsBwOnk42zue\nSJun4iZ2ZTQYGNAjgP4x/uw7mc+qrWdIOppL0tFc+kX7cePwLrpfi4iIiAPLOVfOpztS+fpgFrV1\nNrzdnZgyNIrr+4bi5qI/RUWulf61SJtgMBjoF+1P3+5+HPjmHKu2nmbviXz2nsinT7fOTBveha4h\nXvaOKSIiItfodFYxn25PYfexPGxAgI8riYMjGN4rSPdjE/kBVNykTTEYDCR060zvrr4cTilg1Ven\nST51luRTZ+nVxZdpw6OIDutk75giIiJyCTabjcMpBazZlsKRlAIAIoM8mTwkkgEx/hiNmhIp8kOp\nuEmbZDAYiI/yJT7Kl2OpBXy89QwHT5/j4Olz9Iz04cbhUfSI8LF3TBEREaF+pP/u43ms2Z5CSva3\nI/0nDYkkLtJH92ETaQYqbtLm9Yjw4ZEIH06kF7KqocAdSSkgJsybaSO66BeCiIiIndTU1rH1YDZr\ndzSM9KfpSH8RaT4qbuIwosM68V+39eVUZhGrtp5h/6mzPPeffXQL9WLasC707uqrAiciItIKyitr\n2bwvg892pVHUMNJ/dN8QEgdFEKj7soq0CBU3cTjdQrx58JY+nMkuZtXWM+w9kc8LS5OJCvLkxuFd\nGO+n2wiIiIi0hMLSKj5LSmPzXo30F2ltKm7isKKCvPjNzATScktZ9fUZdh/NZdGH+/lkWwqTBofT\nL8Yfo87AiYiI/Gg5BeWs3ZHK1gP1I/29NNJfpNXpX5o4vPAAD+69uRcZeaV8si2FXUdy+OeKIkL9\n3Zk2LIqBPQI0xUpEROQHOJNdzJrtqew+lovNBgGdXEkcopH+Ivag4ibtRqi/B7+6MZ6fTotn8erD\nbD+Uw6srDxHc+TRTh0UxqGcAJqPR3jFFRETatPMj/T/dnsLhMw0j/QM9mTxUI/1F7EnFTdqdsABP\nfjE1jhuHR7F6WwpfH8zm9VWHWfnVaaYOjWJIfCBmkwqciIjIhS410r9npA+Th2qkv0hboOIm7VaA\njxs/m9yTacOiWLMjlS+TM3ljzRE+3nqaKUMjGd47WAVOREQ6vEuO9O/hz6QhkXQJ1kh/kbZCxU3a\nPb9Ortw1sQdTh0by6fZUtiRn8vbaY6z6+gyTh0QyMiFY6/RFpEWUlZXx6KOPUlRURE1NDffddx99\n+/bloYceoqioiMDAQJ5//nmcnJzsHVU6oPLKWrbsy2D9BSP9R/UJIXFwBEEa6S/S5qi4SYfh6+XC\nnBtimDIskrU7Utm8N4N31x/nk6/PMGlIJKP7hOBkUYETkeazYsUKunTpwm9/+1tycnKYO3cuY8eO\nZcSIEfz0pz/l5Zdf5ujRoyQkJNg7qnQgRaVVrL9gpL+Lk4lJgyOYcJ1G+ou0ZSpu0uF08nBm9rho\nJg2JZN3OVDbtyeDfG06welsKiYMiGNMvFGcnFTgR+fF8fHw4duwYAMXFxfj4+LBp0ybeffddAObN\nm2fPeNLBZOaX8v7ao2w9kE1tnRUvdycmD4lkTL9Q3Fws9o4nIldhsNlsNnuHAMjLK/nRn8Pf37NZ\nPk9rcKSs4Fh5v2/WkvJq1u9KY+PudCqr6/BwtTBxUDhj+4fh6tyyr2205+fV3hwpb0fL6u/v2Uxp\nHMPPf/5zUlNTKS4u5rXXXuOXv/wlc+fO5euvv6Z79+488cQTV1wqWVtbh1nLueVHOJleyIefn+Dr\n/ZlYbRDc2Z3pY7ozbmC4VpqIOBCdcZMOz9PNiZmjuzFxUAQbktL4LCmdD7d8w9odqdxwXTjjBoTr\n5qIi8oOsXLmSkJAQ/vWvf3H06FHmz59PVVUVw4cPZ968eTzxxBMsXbqUOXPmXPZzFBSUN0uWjvYC\nQWtpq1ltNhtHGkb6H2oY6d811JsbBoY13t+0qLB5frZaSlt9bi9FWVuGI2WFln9xU3+NijTwcLVw\n88iu3HBdBBv3pLN+ZyorvjzN2p1pTBgYxviB4Xi4aimJiFy7PXv2MGLECABiY2PJzc0lKCiIfv36\nATB8+HB27Nhhz4jSzlitNvY0jPQ/c+FI/yGRjL4ugvz8UjsnFJEfSsVN5DvcXMxMGxbF+AFhbNqb\nwbqdqXy89Qzrd6UxbkAYN1wXjqebJsCJyNVFRkaSnJzMxIkTycjIwN3dncGDB7N9+3aGDBnCoUOH\n6NKli71jSjtQU2vl64NZrN2RSk7DSP8BPfyZfMFIf92HTcSxqbiJXIars5nJQyIZ1z+Mzfsy+HRH\nKqu3pbAhKZ0x/UOZOCgCb3cVOBG5vNtuu4358+dzxx13UFtbyx//+Ed69OjBww8/zKJFi/Dz8+Pe\ne++1d0xxYBVVtWze+92R/sEkDo7USH+RdkbFTeQqnJ1MTGyYNvlFciZrtqewdkcqn+9OZ3TfUBIH\nR+DjqfHJInIxd3d3XnzxxYu2v/HGG3ZII+1JUWkVnyWls2lvBhVVtbg4mUgcHMGEgeH6nSTSTqm4\niVwjJ4uJ8QPDGd03hK/2Z7FmewqfJaWxaW8Go/oEM3lIJL5eLvaOKSIi7VhOQTnrdqTy1fmR/m4W\nJo/uqpH+Ih2AipvI92QxmxjTP4yRfUL4+mA2n3x9hs/3ZLBlXyYjE+oLnF8nV3vHFBGRdiQlu4Q1\n21NIOpaLzQb+nVxIHBzJ8F5BGukv0kGouIn8QGaTkVF9QhjWK4gdh3P45OszbN6XyZf7sxjaK4gp\nQyMJ9NH1BSIi8sNcaqR/RKAHk4dEMqCHPyaj0c4JRaQ1qbiJ/Ehmk5HhvYMZEh/IriO5rPr6DF/t\nz2LrgSyGxAUxdVgkwZ3d7R1TREQcxOVG+k8aEkF8lK+mQ4p0UCpuIs3EZDQyJD6IQT0DSTpWX+C2\nHcpm+6FsBsUFMnVoJKH+HvaOKSIibVRNrZVth7L5dHvKZUf6i0jHpeIm0syMRgODegYyMDaAvcfz\nWbX1NDsO57DjcA4De/gzdVgUEYGe9o4pIiJtREVVLZv3NYz0L63GZKwf6T9xUIRWbIhIIxU3kRZi\nNBgY0MOf/jF+JJ86y6qtp0k6lkfSsTz6RfsxbXgU/v4qcCIiHVVRWTUbktL4fE/9SH9njfQXkStQ\ncRNpYQaDgb7d/ejTrTMHT5/j462n2Xsin70n8hmWkMFPxnbHXSOcRUQ6jNyCctbuTOOr/VmNI/0n\njerKmP6h+n0gIpel4ibSSgwGA727dqZXF1+OpBTw0Zen+Xp/FidTC5k3ozdhAbr+TUSkPUvJLuHT\nHSnsOqqR/iLy/am4ibQyg8FAXJQvsRE+rN+TwQcbjvPXxUn8bFJPBscF2jueiIg0I5vNxtGUAtbs\nSOXQ6XMARAR4MHmoRvqLyPej4iZiJ0ajgTsn9cTf05l/rT7Max8f4nRWMbOu74bZpF/kIiKO7PxI\n/093pHA6q36kf2xEJyYPjdRIfxH5QVTcROxsQA9/QvwG8vLyA6zflUZKdgn33NwLb3cne0cTEZHv\nqXGk/45Ucs6V14/0j/Fn0pBIuoZopL+I/HAqbiJtQHBnd564ayBvrD7C7uN5/PmtXdw7vRfdQrzt\nHU1ERK7BpUb6j0wIJnGwRvqLSPNQcRNpI1ydzdw7vRef7kjlwy2n+Nt7e5gzIYbRfUPtHU1ERC6j\noKSSD7ecajrSf1AEE67TSH8RaV4qbiJtiMFgYPKQSCIDPXl15UHeXnuM01nFzJnQA4tZ172JiLQl\na7ansPKr09TUWvF0szBDI/1FpAWpuIm0QfFdfFnw0+t4ecUBvkjOIi23lPum98bXy8Xe0UREBNi8\nN4Nlm0/R2duFyYMjGN47WCP9RaRF6SV8kTbKr5Mr8+8YwLBeQZzOKuFPb+3iSEqBvWOJiHR4+07m\ns3j9MTxcLSy8dzhj+oeptIlIi1NxE2nDnCwmfj6lJ3fcEEN5ZS3P/Wcf63amYrPZ7B1NRKRDOp1V\nzKsrD2IxGXnglgRC/DzsHUlEOggVN5E2zmAwMLZ/GL+7vR+e7haWfH6S1z4+RFV1nb2jiYh0KLmF\nFby4NJmaWiu/ujFek39FpFVd0zVuCxcuJDk5GYPBwPz580lISGj82AcffMCyZcswGo3ExsayYMEC\ndu7cyQMPPEB0dDQAMTExPPnkky3zHYh0ENFhnVjw0+t45aOD7DySS0Z+GfNm9CbQx83e0URE2r3S\nihr+8UEyxeU1zJkQQ78Yf3tHEpEO5qrFbefOnaSkpLBkyRJOnTrF/PnzWbJkCQAVFRWsXr2a9957\nD4vFwl133cXevXsBGDRoEIsWLWrZ9CIdTCcPZ373k34s2XiSjXvS+fNbSfxyWhx9u/vZO5qISLtV\nXVPHomX7yTlXzqTBEYwbEGbvSCLSAV11qeS2bdsYP348AN26daOoqIjS0lIAXF1defvtt7FYLFRU\nVFBaWoq/v16BEmlJZpOR/9/enYdHVd97HH/PZCb7vickQAhLIBCWAC6ssilLrwpCtSpaUavAtXq1\nt5YWsV711pbrrgWsLfuRskAAACAASURBVOL1FgQjYK1iAUU2lUVDCCD7koVsZN8nmftHkoGwJAGS\nzAz5vJ4nz8ycM3Pmk98D85tvzvecc/eEnsya3BtLTS2vr9rD6s1HqdVxbyIira621so7/9jH4fRC\nhvYOZdroWHtHEpEOqtk9brm5ucTHx9seBwYGkpOTg7f32YNxlyxZwrJly5g5cybR0dFkZGRw+PBh\nHnnkEQoLC5k7dy7Dhg1r8n0CAjwxma7+jEwhIT5XvY324kxZwbnydoSst43xoV/PUF58bwdrtx4n\n40w5T96diLdH210/yJnGFZwrr7KKOKYVGw+z68ccekX7M2tyH4wGg70jiUgHddnXcbvY2ewefvhh\nZs6cyUMPPURiYiJdu3Zl7ty5TJw4kVOnTjFz5ky++OILXF1dL7nd/Pyyy41ygZAQH3Jyiq96O+3B\nmbKCc+XtSFl93Vz43b2JLF6bys79Wfxy4ZfMndqPqNDWP8uZM40rOFfejpZVhZ84iy92nOJfO08R\nGezF3Gn9MJt0TjcRsZ9mP4FCQ0PJzc21Pc7Ozra1QxYUFLBjxw4A3N3dGTlyJLt37yYsLIxJkyZh\nMBjo3LkzwcHBZGVltdGvINKxeXuYeWJ6fybf0IXsgnKef38n3+7T/zcRkaux80A2KzYcws/Llcen\nJ+Dl3nbdDCIiLdFs4TZs2DDWrVsHQGpqKqGhobY2SYvFwtNPP01paSkAKSkpxMTEsHbtWt59910A\ncnJyyMvLIywsrK1+B5EOz2g0MG1ULHNu74fRYGDx2lSWbzhETW2tvaOJiDidQ2kFLPlkH66uLjw+\nvT/Bfh72jiQi0nyr5KBBg4iPj+fOO+/EYDCwYMECkpKS8PHxYfz48cyZM4eZM2diMpno1asXY8eO\npbS0lKeeeooNGzZQXV3Ns88+22SbpIi0jsReIUQGD+bNpBS+2HGKk1nFPHJrX3y99P9PRKQlMvNK\neX3VHmprrTw2rR9dwtXaKyKOoUXHuD311FONHsfFxdnuT506lalTpzZa7+3tzaJFi1ohnohcrogg\nL343czDvfrqf3Qdz+P3SHcy5vR/dIn3tHU1ExKEVllbxyofJlFZY+PnEOPp2C7J3JBERGx1lK3IN\n8nAzMef2vkwb1Y2Ckkr+8MEuvk7OsHcsERGHVVlVw2srk8ktrODfhnVlRP9Ie0cSEWlEhZvINcpg\nMDD5hq48MaM/bmYXln52gKWfHaDaouPeRETOVVNby6I1ezl+uphh/cK5dXiMvSOJiFxAhZvINa5v\nTBAL7h9C5zBvvk7O4A8f7OZMUYW9Y4mIOASr1coH/zpE8pE84rsGcN8tcRh0rTYRcUAq3EQ6gGB/\nD+bdk8gN8eEcyyzi90t3cOBEvr1jiYjY3T+/OcFX36cTHerN7Nv7YXLRVyMRcUz6dBLpIFzNLjw4\npTd3j+9JWYWFhct/YN13J7FarfaOJiJiF9+knuajTUcJ8HHj8en98XBr0TnbRETsQoWbSAdiMBgY\nmxjFf/5sID6eZlZsPMzitalUVtXYO5qISLvafyKfdz/dj4ebiSdm9CfAx83ekUREmqTCTaQD6hHl\nz4KfD6F7lB/f7c/m+fd3kpVfZu9YIiLtIi2nhDeTUgCYO7UfUSHedk4kItI8FW4iHZS/txv/eddA\nxg6KIj2nlOeW7iT5cK69Y4mItKn84kpeXZlMeaWFByb3pneXAHtHEhFpERVuIh2YycXI3RN6Mmty\nbyw1tby2ag9rthyjVse9icg1qLzSwqsrkzlTVMm0Ud24IT7c3pFERFpMhZuIMKxfBPPuSSTYz501\nW47xxqo9lFVU2zuWiEirsdTU8vbHKZzKLmH0gEgmXd/F3pFERC6LCjcRAaBLuA/P3D+E+JhAko/k\n8dx7O0nLKbF3LBGRq2a1Wnnv8wOkHs+nf2wQd0/oqWu1iYjTUeEmIjbeHmaemN6fyTd0ITu/nOeX\n7eS7/Vn2jiUiclXWbDnG1pTTdA334ZFb++Ji1NcfEXE++uQSkUaMRgPTRsUy5/a+GAwGFq1JZcXG\nQ9TU1No7mojIZducnMHarccJ9nPnl9P74+bqYu9IIiJXRFeaFJGLSuwVSkSQF28mpbDuu1Nk5JUz\na1Icvl6u9o4mItIie4/m8d7nP+LlXnetNj99fomIE9MeNxG5pMhgL+bfN5hBPUNIOZLL75fu4GhG\nkb1jiYg068TpYt5avRej0cBjdyQQEeRl70giIldFhZuINMnDzcSc2/syc1JvCoor+cMHu/g6OcPe\nsURELimvsIJXVyVTVVXDwz/pQ48of3tHEhG5amqVFJFmGQwGpo/tSbCPK4vXpLL0swMcyyziZ+N6\nYjbp7z8i4jhKK6p5ZWUyhSVVzBgTS3x3Xwori6muraKqpprq2mqqaqqoqq2mqubs/eqaKqpqLfW3\n5z2ntorqmur65zVeZ3Jxwdfsg5+bL/5ufvU/dff96u97m710FksRuWoq3ESkxfrGBPHM/UN4KymF\nTT9kcDKrhDm39yXQ193e0UTEidTU1tgKn4sVVO4VLuTmF9UVSg3Pu0hBde5rqmuqqaypIr+0jJrO\nFjxjavmkpJZPvm693GajCVejK2YXM+4mN3xcvTEYIa8sn4zS05d8nclows/V11bQNRR3fo3u+2Iy\n6muZiFyaPiFE5LKE+Hvwm3sTWfb5j2xPPc3vl+7g0Vv7EtclwN7RROQqNS6o6oqii+2hqq6tPvu8\n8wqqhj1TjR6f95oaa02r5jYbTZiNrlRVgaXaiIfZh0g/H1xdXHF1MdcVW0Yzri5mzPWPXc+5bVjX\nUJRduM4Vs9GE0XBhh0FIiA85OcVU1lRRUFlIQUUhBZWFFFYWUVBVSEFlkW3Z0cITWLFe8vfwMXuf\nU9D54u/mf87eu7pbD5O79t6JdFAq3ETksrmZXXhwSm+6RfqyfMMhFi7/gRk3xTJ+SLS+UIg4mJra\nGtYc+YzSgyUUl5ddUGBVn1OUtXZBZTKacK0vfNxc3PB29T5bHBnPK5Aa7hvNBPh5U1Ve26jAOltQ\nmTE3LD+noFr55WE+23mS7p38eOrOAbia2/e0/24uroR5hhDmGXLJ59TU1lBcXUJ+RSGFlfVFne22\ngMLKIrLKcjhVcunjiF1dXOuKOde6VswAd79z2jTrbn1dfS5aZIqIc1PhJiJXxGAwMDYxis5h3rz9\n8V6WbzzMsdPF3H9LnK6TJFKvtLSUX//61xQWFlJdXc2cOXMYMWIEAMuXL2fJkiVs3LixTTOUVJex\nKX0bllqLbdnFCyozZhfXCwuqFuyNanjNxQqqK9GwF6ulNuxK47NvTxIW6MljdyS0e9HWUi5GF1ur\n5KVYrVbKLRX1BV1dUVd4zv2G5dlluZfchtFgxNfV54KCLro0DJcqV1uLppuLLo8g4kxUuInIVekR\n5c8z9w/hz6v38u2+LNJzSpgztR9hAZ72jiZidx9//DExMTE8+eSTZGVlcd999/H555+Tl5fHv/71\nr3bJ4Ofmwx+GP4NfgBtF+VW4upivqb0x3x/K4f/WH8TX08wTM/rj7WG2d6SrYjAY8DR74Gn2INI7\n/JLPq661UFRZRH5l4713hecsSy/O4ETRqbMvOtJ4Gx4mj0atmAHnnFClocD0MnteU/9eRK6E1Wql\nutZCZU1l/U9V3a2l7raiphIDBiYE3NimOVS4ichVC/Bx4z9/NpDlGw6xcXc6zy3dycM/6UP/7sH2\njiZiVwEBAfz4448AFBUVERBQdyzon/70Jx577DGeeOKJdsnhYXLHz92HKlPL92I5gyMZhSxek4rZ\nZOSX0/sT6u9h70jtxmw0EeQRSJBH4CWfY7VaKakutbVi1pirSMvLarTnrqCyiMzSrEtuw2Rwwa/R\ncXcXnlzFz80Xs06sIg6iJUWWbdl56yrPW1dhOfu4qeNTG4QHBdDFtVub/W76XyYircLkYuSeCb2I\nifBl2bofeX3VHm4dHsOUYV0x6rg36aAmT55MUlIS48ePp6ioiMWLF/Ptt9/i5uZG//797R3PqWXn\nl/H6qj1U19Ty79MSiInwtXckh2MwGPBx9cbH1Zton8i6FlS/C4v3ypqqi7Rj1rVo5tfvxTtedJJa\na+0l38vb7NWoLdP/InvvdGIVOV9zRZZriZGcgsI2KbKaUtdGXtdK7uPqbbtvuzW5XbDMx9WLARF9\nyc8ra6XRuZAKNxFpVcP6RRAV4s2bSSms3nKMY5lFPPSTPni6O3f7ksiVWLNmDZGRkbz77rscOHCA\n3/zmN3h6evL222+3eBsBAZ6YTK1zzFZIiE+rbKc9NJW1sKSS1//yLcVl1cyelsD4G2LaMdmFroVx\njSKoydfV1tZSUFnEmbICzpSf81P/OK88n5zyPNKaOLGKm4srgR7+BHr6E+DhT5CHv+1xoIc/QR4B\n+Ln74GI8++/9WhhbR3QlWa1WK9U11VRYKqmwVFJuqbDdr7BUUlF9kWWWSiqqL1x27vOs1qsrstxM\nbrib3HB3dcPf5ItHw2OTe/2tG+5mt7P365d7mC98nofJDVeT61W1B7flvwMVbiLS6rqE+7Dg50NY\nvGYvyUfy+K/3djJnaj+iQrztHU2kXe3evZvhw4cDEBcXx5EjRwgPD+ehhx4CIDs7myeeeIJXXnnl\nktvIz2+dv95e7gk/7KmprFXVNfxp+fdk5JYy6fouDO4RbNff61oZ15ZxwY8g/NyCiHED/BuvtVqt\nVNRUNLoEQkH9ZREKbZdKKCKzJPuS72DAYDuxioerKxZLLWCwdW4YMGAwGGy3jZdRv8zQeBkGOOd+\nw+vBgMFge7bt+TTanvEiyxpef/a+l6cb5eXVABht71f/LIOhbtl5mWz5z3muob5gaLys7hV12z2b\n9+x2L/78iy4DvH3dyDlTWN8y2PyerLN7s1p3T1agmydunm64N9qT1bAXq25ZsL8fVWW1uLm44m5y\nr3/e2T1dri5XV2QBYKn7sQDFVAPVV7yp1vg8aKrwU+EmIm3C28PMEzMGkPT1Uf75zQleWLaLn0+K\nY2jvMHtHE2k3Xbp0ITk5mZtvvpn09HQiIiJYt26dbf2YMWOaLNqksdpaK0s+2ceR9CKu7xPG1FFt\ndyyJXD6DwYCHyQMPkwcRXpf+rLfUWiisLKawqvASl0YoJLP0NDUltVit1qsuFuTKnFtkBXl42ool\n94bCynRe+6Dtft1j9/PaCa+kyHKmP4y0BxVuItJmjEYDd4yOpWu4D+/+cz+L1qRyLLOIO0bH4mLU\nWcrk2vfTn/6UefPmcc8992CxWHj22WftHclpWa1Wlm84xO6DOcR19ufnk3rr+FknZTKaCPIIIMgj\nAC59ZYRGX9rPLeAa7lvr72O7X3uRZQ3PtVLXkddw/2LLwErdNjjvPRreu7Zh2+ct8/f3IL+gtH4b\n1nMyWM8uwwpW6t/hwm1ckPmSy86+R90zauGc9zj//c4ur7v18/GkupxWL7Kk7alwE5E2NzgulMhg\nL95MSmHdd6c4cbqYR27ri6+nriEk1zYvLy9ee+21S65v62u4XUu+2HGK9bvS6BTsxdyp/TCb9KWy\nIzm3PREHrNdDgn3IsTrHniHtxXJe+tQTkXYRGezF/PsGM7BHMAdOFvDc0h0cyyyydywRcQI7DmSz\nYuNh/L1deXx6f53sSEQ6JBVuItJuPNxMzJnaj2mjupFfVMl//+8uvk6+9BnIREQOnirgnU/24ebq\nwuPT+xPk527vSCIidqHCTUTaldFgYPINXXliRn/czC4s/ewA731+gGrLpa8PJCIdU2ZeKW98tAer\n1cqc2/vSOcx5TrcuItLaVLiJiF307RbEM/cPoXOoN5t+yOAPH+zmTFGFvWOJiIMoLKnklQ+TKa2w\ncN8tcfSNafo6YyIi1zoVbiJiNyH+Hvzm3kRuiA/nWGYRzy3dwY8n8+0dS0TsrLzSwqur9pBbWMFt\nw2MYnhBh70giInanwk1E7MrN7MKDU3pz9/ielFZY+NPff+CLHadspzwWkY6lpraWP76/kxOnixmR\nEMFPhnW1dyQREYegwk1E7M5gMDA2MYpf3TUQH08zyzccYskn+6isqrF3NBFpR1arlf/94iA792fR\nNyaQe2/uhUHXahMRAVS4iYgD6RntzzP3D6F7Jz++3ZfFC+/vJDu/zN6xRKSdfLr9BJt+yKBbJz8e\nva0vJhd9TRERaaBPRBFxKAE+bvznzwZy06BOpOWU8tzSnew5kmvvWCLSxrbvPU3S10cJ8nVjwYPX\n4+FmsnckERGHosJNRByOycXIvRN68cCk3lRZanlt5R7Wbj1GrY57E7km7T9+hr/+cz+ebiYenzGA\nQF9dq01E5Hwq3ETEYQ1PiOC39yYS6OvO6s3HePOjFMoqLPaOJSKtKC27hDc/TsFggH+f1o9OwV72\njiQi4pBUuImIQ+sS7sMz9w+mT9cAfjicy3+9t4O0nBJ7xxKRVnCmqIJXViZTXlnDrMl96NU5wN6R\nREQclgo3EXF4Pp6u/MeMAUy8vjNZ+eW8sGwX3+3PsncsEbkKZRUWXl2ZTH5xJdNHx3JdnzB7RxIR\ncWgtOvL3xRdfJDk5GYPBwLx580hISLCt+/DDD1m1ahVGo5G4uDgWLFiAwWBo8jUiIpfLaDQwfXR3\nYsJ9efef+1m0JpXjmcVMG90NF6P+BiXiTCw1tbz1cQppOaXcNKgTt1zX2d6RREQcXrOF23fffceJ\nEydYsWIFR44cYd68eaxYsQKA8vJyPv30Uz744APMZjMzZ87k+++/x2KxXPI1IiJXY3BcKBHBXryZ\nlMLn353kRFYxv7g1Hl9PV3tHE5EWsFqtLP3sAPtP5DOgezB3j+upa7WJiLRAs3+m3r59O+PGjQMg\nNjaWwsJCSkrqji/x8PDgvffew2w2U15eTklJCSEhIU2+RkTkanUK9uKZ+wYzsEcw+0/k89zSHRzL\nLLJ3LBFpgY83H2Pb3tPERPjyi1vjMRpVtImItESze9xyc3OJj4+3PQ4MDCQnJwdvb2/bsiVLlrBs\n2TJmzpxJdHR0i15zvoAAT0wmlyv9PWxCQnyuehvtxZmygnPlVda24WhZn334RlZtPMT/fr6fP3yw\nm0enJjD+ui629Y6WtynKKh3Bph/S+ce244T4u/PLOxJwM1/9vC8i0lFc9tUtrRe5jtLDDz/MzJkz\neeihh0hMTGzRa86Xn192uVEuEBLiQ05O8VVvpz04U1ZwrrzK2jYcNetN/SMI8XFl8dpUXv/wB/Yc\nzOaucT2JjPBzyLwX46hjezGtkVWFX8e050ge7687iLeHmf+YMQBfL7U3i4hcjmZbJUNDQ8nNzbU9\nzs7OJiQkBICCggJ27NgBgLu7OyNHjmT37t1NvkZEpLX17RbE/PuHEB3qzVc/ZPDS/+3m4Ml8e8cS\nkXrHTxfx59V7cXEx8NgdCYQFeto7koiI02m2cBs2bBjr1q0DIDU1ldDQUFvLo8Vi4emnn6a0tBSA\nlJQUYmJimnyNiEhbCPX3YN69idwQH8bRjCKefO1rXli2k2/2ncZSU2vveCIdVm5BOa+u3ENVdQ0P\n/ySe7p387B1JRMQpNdsqOWjQIOLj47nzzjsxGAwsWLCApKQkfHx8GD9+PHPmzGHmzJmYTCZ69erF\n2LFjMRgMF7xGRKStuZldeHBKH27sF8HmPafZse80R9buY4X3YW4a2IlRAzrhp/YskXZTUl7NKyuT\nKSqt4q5xPUjspe4bEZEr1aJj3J566qlGj+Pi4mz3p06dytSpU5t9jYhIezAYDMR3DWT0kC6kHsxi\n4+50Nu/JYPXmY/xj23GGxIUxbnAUMRG+9o4qck2rttTw5kd7yMwrY8KQaMYPjrZ3JBERp3bZJycR\nEXEWoQGe3Dm2B7eNiGHb3tNs2JXG9tTTbE89TWwnX8YlRpPYKwSTiy7gLdKaaq1W/vKP/RxMK2RI\nXCgzxnS3dyQREaenwk1ErnnuribGDIpi9MBO7Dt+hg0709hzJI/F6an4e7va2ih1ljuR1rHqyyPs\nOJBNzyg/HpzSG6MusC0ictVUuIlIh2E0GOgbE0TfmCCy8svYuCudLSkZfLz5GJ9sO851vcMYOziK\nruFqoxS5Uht2pfH5dyeJCPJk7rQEzK1wjVYREVHhJiIdVFiAJ3eNO9tGuX5XGlv3nmbr3tN0j/Jj\nXGIUg3qqjVLkcuw+mMP//esgvl6uPDG9P94eZntHEhG5ZqhwE5EOzcPNxNjEKG4a1Il9x86wfldd\nG+XhtEICfNwYPbATowZE4uupNkqRphxJL2Tx2lRczS48Pj2BYH8Pe0cSEbmmqHATEaG+jbJbEH27\nBZF1powNu9LYkpLJx18f5ZOtx7muTyjjEqPpEu5j76giDicrv4zXVu2hpsbKnDv6qt1YRKQNqHAT\nETlPWKAnPxvfk9tHdmNrSiYbdqWxNeU0W1NO0yPKj7FqoxSxKSqr4pUVyZSUV3PfLb1IiA22dyQR\nkWuSCjcRkUvwcDMxbnA0YxKj2Hv0DBt2pZFyNI9D9W2UNw3sxEi1UUoHVlldw+ur9pBdUM6UG7sw\nakAne0cSEblmqXATEWmG0WAgITaIhNggTp/TRpn09VHWbj3O9X3qLurdOUxtlNJx1NZaWbI2laMZ\nRdwQH87tI7rZO5KIyDVNhZuIyGUID/Tk7vE9mTqyG1vq2yi3pGSyJSWTnlF+jBsczcCewbgY1UYp\n1y6r1crf1x/i+0O59O4SwM8nxWHQtdpERNqUCjcRkSvg4WZi/OBoxiZGsfdoHut3prH32BkO1rdR\njhnUiZH9I/FRG6Vcg9Z9d4oNu9OICvFizu39dLyniEg7UOEmInIV6toog0mIDSYzr7TuRCZ7T/PR\nprNtlGMT1UYp147v9mfx4ZeHCfBx4/Hp/fF011cJEZH2oE9bEZFWEhHkxT0TejF1ZKztbJSb92Sy\neU8mPaP9GZcYpTZKcWo/nsznL//Yh4ebC49P70+gr7u9I4mIdBgq3EREWpmnu4nxQ6IZOziKlCN5\nrN+VRuqxMxw8VUCgrxtjBkUxsn8k3h5me0cVabH03FLe+CgFqxXm3N6P6FBve0cSEWkXX321gdGj\nxzb7vBdeeIEpU6YRGdk2Z9hV4SYi0kaMBgP9uwfTv3swGbmlbNidxraU06z66ghrthzjhvgwxiZG\nExKiNkpxbAUllbz64Q+UVVqYNbk3fboG2juSiEi7yMzMYP36dS0q3H7729+Sk1PcZllUuImItIPI\nYC/undCLaSO7sSXlNBt2neLr5Ey+Ts6kX2wwIxPCGdBDbZTieMorLby6Mpm8okpuHxHDsH4R9o4k\nItJuXn75JfbvT2XEiCFMmDCRzMwMXn31bf77v58jJyeb8vJyHnjgYYYNG8G9997L3Ln/wZdfbqC0\ntISTJ0+Qnp7GY489yQ03DLvqLCrcRETakae7mQlDohmXGMWeo3ls2HmKlCO5pBzJJai+jXKE2ijF\nQVhqavnzmr2czCphZP8IptzY1d6RRKSD+nDjYXYcyG7VbQ6JC2XGmO5NPueuu+4lKelDYmJiOXny\nOG+//Rfy888wdOj1TJw4hfT0NObPf5phw0Y0el12dhYLF77ON99sY82aj1S4iYg4K6PRwIDuwQzo\nHkx5jZVV6w+ydW8mK+vbKK+PD2dcYhRROo5I7MRqtfL+uh/Ze/QM/boFce/NvXStNhHp0Hr3jgfA\nx8eX/ftTWbs2CYPBSFFR4QXPTUgYAEBoaCglJSWt8v4q3ERE7KxzuC/33tyLqaO6sWVP3dkov07O\n4OvkDOI6+zNucDQDugdjNOpLs7SfT7YdZ/OeTLqE+fDobfFq4xURu5oxpnuze8famtlc1w3zr399\nTlFREW+99ReKiop48MF7L3iui4uL7b7Vam2V91fhJiLiILzczdw8tDPjB0eTfCSX9TvT2H8inwMn\nCwjydWdsYhQj+kfg5a42SmlbW1MyWb35GEG+7jw+PQF3V31dEJGOyWg0UlNT02hZQUEBERGRGI1G\nNm3aSHV1dbtk0SexiIiDMRoNDOwRwsAeIaTnlLBhVxrb9p7mwy8Ps3rzUW7oG87YxCiiQtRGKa0v\n9dgZln52AC93E0/M6I+ft5u9I4mI2E2XLjH8+OMBIiIi8ff3B2D06DE8/fR/sG/fXiZP/jdCQ0P5\n29/eafMsKtxERBxYpxBvZt4Sx7TRsWxOzmTj7jQ2/ZDBph8y6N0lgHGJUfRXG6W0kpNZxbz1cQoG\ng4F/n5ZAZLCXvSOJiNhVQEAASUmfNloWERHJe+8ttz2eMGEiACEhPuTkFNOt29mWzm7duvPmm0ta\nJYsKNxERJ+DlbuaW6zozYUg0yYdzWb+rro1y/4l8gv3c689GqTZKuXJniip4dWUyFVU1PHJrPD2j\n/e0dSUREzqHCTUTEiRiNBgb2DGFgzxDS6tsotze0UW45yo19IxibGEUn7SmRy1BWUc0rK5MpKKli\nxk3dGdo7zN6RRETkPCrcREScVFSIN/fdEse0UbFs3pPBxl3pfPV93U/vLgGMGxxF/1i1UUrTLDW1\nvJmUQnpOKWMTo7h5aLS9I4mIyEWocBMRcXLeHmYmXteFm4d05ofDuazfeapRG+XYxChGJETgqTZK\nOY/VauVv/9zPgZMFDOoZwl1je+habSIiDkqFm4jINcJoNDCoZwiDeoaQll3C+l1pbE89zYqNh/l4\n81GG1bdR6oQT7ae0tJRf//rXFBYWUl1dzZw5cwgJCeG5557DaDTi6+vL//zP/+Dh4WGXfElfH2V7\nahaxkb48/JM+2jsrIuLAVLiJiFyDokK9uX9iHHeMjmVzcgYbd6fx5ffpfPl9OvFdAxibGE1CbJC+\nqLexjz/+mJiYGJ588kmysrK47777CA4O5umnnyYhIYGXXnqJpKQk7r777nbP9tUP6Xy6/QShAR78\n+x0JuJpdmn+RiIjYjQo3EZFrmLeHmYnXd2HC0Gh+OFR3Ue/U4/mkHs8nxN+dsYOiGK42yjYTEBDA\njz/+CEBRUREBdquC3AAADI1JREFUAQEsWrQIb++6a/AFBgZSUFDQ7rmSD+fy/rof8fYw88SM/vh6\nurZ7BhGRa8Udd/yEZctWAD5t+j4q3EREOgAXo5HEXqEk9grlVHYJG3adYntqFss3Hubjzce4sV84\n4xKjiAhSG2Vrmjx5MklJSYwfP56ioiIWL15sK9rKyspYs2YNr732WrtmOpZZxJ/X7MXsYuSX0xMI\nC/Bs1/cXEZEro8JNRKSDiQ715v6JvbljdHe+bmij3J3Ol7vTiY8JZFxiFP1igzDqJBVXbc2aNURG\nRvLuu+9y4MAB5s2bR1JSEmVlZTz66KM88MADxMbGNrmNgABPTKbWaWOsMRp546MULJZafnP/UK7v\nG9Eq220LISFt+5fr1qSsbceZ8ipr27Bn1ttvv5233nqLyMhI0tPTmTNnDmFhYZSVlVFRUcH8+fNJ\nSEjAxcVIcLB3m+dV4SYi0kF5e5iZdH0Xbh4azfcH6y7qnXrsDKnHzhDq78HYxCiG9YvA011TxZXa\nvXs3w4cPByAuLo7s7GyqqqqYPXs2U6ZMYerUqc1uIz+/rFWyuHm6MX/RNgpKKrl7fE9iw7zJySlu\nlW23tpAQH4fNdj5lbTvOlFdZ28a5WZMO/4Pvs1NadfsDQ/sxtfuUS66/8caRrF37GdOmzWDNmn9y\n440jiY3twciRo9m1awdvvvk2L7zwJ2pqasnNLcHLy+uqx7apws94VVsWERGn52I0MjgulKfvHsSz\nPx/C8IQIzhRX8vcNh3jy7a188MVBMvNK7R3TKXXp0oXk5GQA0tPT8fLy4t1332Xo0KFMnz693XJU\nW2p4/q/fcvpMGbdc15mxiVHt9t4iIs5q5Mib2Lp1MwBbtmxi+PBRbNq0gUcfncWf//wGhYWF7ZpH\nf0YVERGbzmE+PDCpN9NHx9a3UaazYXcaG3an0TcmkJ//W18CPDR1tNRPf/pT5s2bxz333IPFYuHZ\nZ5/lV7/6FVFRUWzfvh2A6667jrlz57ZZBqvVyjv/2M/+42cY2juUO0Y33ZopIuKIpnaf0uTesbbQ\nrVsseXk5ZGWdpri4mM2bvyI4OJT58/+LAwf28eabr7ZrHs2+IiJyAR9PVybf0JWbh3bm+0N1F/Xe\ne+wMf1mzl1/dOcDe8ZyGl5fXBScf2bJlS7tmyC+uZOeBbOK7BTFrch8duygichluuGE4S5a8zYgR\noygoyCc2tgcAmzZ9icViadcsKtxEROSSTC5GhsSFMiQulPTcUiLDfcFSY+9YchkCfd2Zf99g+vcO\np6igdY6XExHpKEaNuolHHnmApUv/TkVFOc8/v4Avv1zPtGkzWL/+Cz79dG27ZVHhJiIiLdIp2IuQ\nAE+nOahdzoqJ8MVNF9gWEblsvXvHs2nTt7bHH3ywynZ/+PBRAEye/G/tkkUnJxEREREREXFwKtxE\nREREREQcnAo3ERERERERB6fCTURERERExMG16OQkL774IsnJyRgMBubNm0dCQoJt3TfffMPLL7+M\n0WgkJiaGF154gR07dvDLX/6SHj3qTpfZs2dP5s+f3za/gYiIiIiIyDWu2cLtu+++48SJE6xYsYIj\nR44wb948VqxYYVv/zDPPsGzZMsLDw3nsscfYvHkz7u7uDB06lNdff71Nw4uIiIiIiHQEzbZKbt++\nnXHjxgEQGxtLYWEhJSUltvVJSUmEh4cDEBgYSH5+fhtFFRERERER6ZiaLdxyc3MJCAiwPQ4MDCQn\nJ8f22NvbG4Ds7Gy2bt3KqFF11zM4fPgwjzzyCHfddRdbt25t7dwiIiIiIiIdxmVfgNtqtV6wLC8v\nj0ceeYQFCxYQEBBA165dmTt3LhMnTuTUqVPMnDmTL774AldX10tuNyDAE5Pp6i8OGhLic9XbaC/O\nlBWcK6+ytg1nygrOlVdZRUREpCnNFm6hoaHk5ubaHmdnZxMSEmJ7XFJSwkMPPcTjjz/O8OHDAQgL\nC2PSpEkAdO7cmeDgYLKysoiOjr7k++Tnl13xL9EgJMSHnJziq95Oe3CmrOBceZW1bThTVnCuvB0t\nqwo/ERGRy2ewXmwX2jl2797NG2+8wd/+9jdSU1N5/vnn+fvf/25b/7vf/Y4hQ4Zw66232patXbuW\nnJwcZs2aRU5ODjNmzGDdunVN7nETERERERGRi2u2cANYuHAhO3fuxGAwsGDBAvbt24ePjw/Dhw9n\nyJAhDBw40PbcKVOmMHnyZJ566imKioqorq5m7ty5tmPfRERERERE5PK0qHATERERERER+2n2rJIi\nIiIiIiJiXyrcREREREREHJwKNxEREREREQenwk1ERERERMTBXfYFuB3Biy++SHJyMgaDgXnz5pGQ\nkGBbt23bNl5++WVcXFwYOXIkc+bMsWPSOk3lHTNmDOHh4bi41F18fOHChYSFhdkrKgcPHmT27Nnc\nf//93HPPPY3WOeLYNpXX0cb2j3/8I7t27cJisfCLX/yCCRMm2NY52tg2ldWRxrW8vJynn36avLw8\nKisrmT17NjfddJNtvSONa3NZHWlcG1RUVDBlyhRmz57N1KlTbcsdaVzlQs40RzrT/AjONUdqfmwb\nmh/bhubIFrI6mW+//db68MMPW61Wq/Xw4cPWGTNmNFo/ceJEa0ZGhrWmpsZ61113WQ8dOmSPmDbN\n5b3pppusJSUl9oh2gdLSUus999xj/d3vfmd9//33L1jvaGPbXF5HGtvt27dbH3zwQavVarWeOXPG\nOmrUqEbrHWlsm8vqSOP66aefWpcsWWK1Wq3WtLQ064QJExqtd6RxbS6rI41rg5dfftk6depU60cf\nfdRouSONqzTmTHOkM82PVqtzzZGaH9uG5se2ozmyZZyuVXL79u2MGzcOgNjYWAoLCykpKQHg1KlT\n+Pn5ERERgdFoZNSoUWzfvt2ecZvM62hcXV155513CA0NvWCdI45tU3kdzZAhQ3jttdcA8PX1pby8\nnJqaGsDxxraprI5m0qRJPPTQQwBkZmY2+uubo41rU1kd0ZEjRzh8+DCjR49utNzRxlUac6Y50pnm\nR3CuOVLzY/tndTTOND+C5siWcrpWydzcXOLj422PAwMDycnJwdvbm5ycHAIDAxutO3XqlD1i2jSV\nt8GCBQtIT08nMTGRJ598EoPBYI+omEwmTKaL/5NwxLFtKm8DRxlbFxcXPD09AVi1ahUjR4607e53\ntLFtKmsDRxnXBnfeeSenT59m0aJFtmWONq4NLpa1gSON60svvcT8+fNZvXp1o+WOOq5Sx5nmSGea\nH8G55kjNj21D82Pb0xzZNKcr3M5ndbLrh5+f97HHHmPEiBH4+fkxZ84c1q1bxy233GKndNcWRxzb\n9evXs2rVKv7617/aNUdLXCqrI47r8uXL2b9/P7/61a9Yu3at3SfKplwqqyON6+rVqxkwYADR0dF2\neX9pPc40R2p+bD+OOLaaH9uGM82PoDmyOU7XKhkaGkpubq7tcXZ2NiEhIRddl5WVZfc2gabyAtx2\n220EBQVhMpkYOXIkBw8etEfMZjni2DbH0cZ28+bNLFq0iHfeeQcfHx/bckcc20tlBcca171795KZ\nmQlA7969qamp4cyZM4DjjWtTWcGxxvWrr75iw4YNzJgxg5UrV/L222+zbds2wPHGVRpzpjnyWpkf\nwfHGtjmONraaH1ufM82PoDmypZyucBs2bBjr1q0DIDU1ldDQUFtbRVRUFCUlJaSlpWGxWPjyyy8Z\nNmyYPeM2mbe4uJhZs2ZRVVUFwI4dO+jRo4fdsjbFEce2KY42tsXFxfzxj39k8eLF+Pv7N1rnaGPb\nVFZHG9edO3fa/uKZm5tLWVkZAQEBgOONa1NZHW1cX331VT766CM+/PBDpk+fzuzZs7nxxhsBxxtX\nacyZ5shrZX4Exxvbpjja2Gp+bBvOND+C5siWMlidqY+i3sKFC9m5cycGg4EFCxawb98+fHx8GD9+\nPDt27GDhwoUATJgwgVmzZtk5bdN533vvPVavXo2bmxt9+vRh/vz5dtuNvXfvXl566SXS09MxmUyE\nhYUxZswYoqKiHHJsm8vrSGO7YsUK3njjDWJiYmzLrrvuOnr16uVwY9tcVkca14qKCn7729+SmZlJ\nRUUFc+fOpaCgwCE/D5rL6kjjeq433niDTp06ATjkuMqFnGmOdJb5EZxrjtT82DY0P9ovryON7bna\ne450ysJNRERERESkI3G6VkkREREREZGORoWbiIiIiIiIg1PhJiIiIiIi4uBUuImIiIiIiDg4FW4i\nIiIiIiIOToWbiIiIiIiIg1PhJiIiIiIi4uBUuImIiIiIiDi4/wdHFGU3IiFTgQAAAABJRU5ErkJg\ngg==\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7fe34bd1cf98>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "wYGUHK0GsfhP",
        "colab_type": "code",
        "outputId": "7997fa4f-1633-4137-886b-ac10c5a31e4b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "# Test performance\n",
        "trainer.run_test_loop()\n",
        "print(\"Test loss: {0:.2f}\".format(trainer.train_state['test_loss']))\n",
        "print(\"Test Accuracy: {0:.1f}%\".format(trainer.train_state['test_acc']))"
      ],
      "execution_count": 83,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test loss: 0.47\n",
            "Test Accuracy: 84.1%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "GO8v_uUisfd9",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Save all results\n",
        "trainer.save_train_state()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "VbQdvtK3twoW",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "## Freeze embeddings"
      ]
    },
    {
      "metadata": {
        "id": "XrMm4K61t21h",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Now we're going to freeze our GloVe embeddings and train on the supervised task. The only modification in the model is to turn on `freeze_embeddings`:\n",
        "\n",
        "```python\n",
        "if freeze_embeddings:\n",
        "    self.embeddings.weight.requires_grad = False\n",
        "```"
      ]
    },
    {
      "metadata": {
        "id": "ilV_QbtktzH3",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "args.freeze_embeddings = True"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "7NdD-iP6tzFQ",
        "colab_type": "code",
        "outputId": "52f0faa1-9ec2-44de-8767-71b9e6f31c2c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "cell_type": "code",
      "source": [
        "# Initialize model\n",
        "model = NewsModel(embedding_dim=args.embedding_dim, \n",
        "                  num_embeddings=len(vectorizer.title_vocab), \n",
        "                  num_input_channels=args.embedding_dim, \n",
        "                  num_channels=args.num_filters, hidden_dim=args.hidden_dim, \n",
        "                  num_classes=len(vectorizer.category_vocab), \n",
        "                  dropout_p=args.dropout_p, pretrained_embeddings=embeddings,\n",
        "                  freeze_embeddings=args.freeze_embeddings,\n",
        "                  padding_idx=vectorizer.title_vocab.mask_index)\n",
        "print (model.named_modules)"
      ],
      "execution_count": 86,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<bound method Module.named_modules of NewsModel(\n",
            "  (embeddings): Embedding(3407, 100, padding_idx=0)\n",
            "  (conv): ModuleList(\n",
            "    (0): Conv1d(100, 100, kernel_size=(2,), stride=(1,))\n",
            "    (1): Conv1d(100, 100, kernel_size=(3,), stride=(1,))\n",
            "    (2): Conv1d(100, 100, kernel_size=(4,), stride=(1,))\n",
            "  )\n",
            "  (dropout): Dropout(p=0.1)\n",
            "  (fc1): Linear(in_features=300, out_features=100, bias=True)\n",
            "  (fc2): Linear(in_features=100, out_features=4, bias=True)\n",
            ")>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "6NJhm2EOtzCV",
        "colab_type": "code",
        "outputId": "775c7f44-2346-459f-a0dd-d31244330b87",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "cell_type": "code",
      "source": [
        "# Train\n",
        "trainer = Trainer(dataset=dataset, model=model, \n",
        "                  model_state_file=args.model_state_file, \n",
        "                  save_dir=args.save_dir, device=args.device,\n",
        "                  shuffle=args.shuffle, num_epochs=args.num_epochs, \n",
        "                  batch_size=args.batch_size, learning_rate=args.learning_rate, \n",
        "                  early_stopping_criteria=args.early_stopping_criteria)\n",
        "trainer.run_train_loop()"
      ],
      "execution_count": 87,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[EPOCH]: 00 | [LR]: 0.001 | [TRAIN LOSS]: 0.53 | [TRAIN ACC]: 80.9% | [VAL LOSS]: 0.48 | [VAL ACC]: 82.6%\n",
            "[EPOCH]: 01 | [LR]: 0.001 | [TRAIN LOSS]: 0.45 | [TRAIN ACC]: 83.5% | [VAL LOSS]: 0.46 | [VAL ACC]: 83.2%\n",
            "[EPOCH]: 02 | [LR]: 0.001 | [TRAIN LOSS]: 0.42 | [TRAIN ACC]: 84.8% | [VAL LOSS]: 0.46 | [VAL ACC]: 83.4%\n",
            "[EPOCH]: 03 | [LR]: 0.001 | [TRAIN LOSS]: 0.38 | [TRAIN ACC]: 86.0% | [VAL LOSS]: 0.46 | [VAL ACC]: 83.8%\n",
            "[EPOCH]: 04 | [LR]: 0.001 | [TRAIN LOSS]: 0.32 | [TRAIN ACC]: 88.1% | [VAL LOSS]: 0.47 | [VAL ACC]: 83.8%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "oojDHLowty_9",
        "colab_type": "code",
        "outputId": "fffbfaf7-fa75-43d9-e70a-d3200ac80013",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 335
        }
      },
      "cell_type": "code",
      "source": [
        "# Plot performance\n",
        "trainer.plot_performance()"
      ],
      "execution_count": 88,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA24AAAE+CAYAAAD1QEO5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xl4lOWh/vHvbNkXErKThRCWQMgC\nhBB2ZZGwKKJW9EhLqz2nHrG2Fvuj0sX2eI62p9LW5XiqHmut2orFsAgIAgKyBUhCwhp2CITsK1nI\nOr8/wCiCLJLkzST357q4MvPmnZl7JkDmnud9n8dkt9vtiIiIiIiISKdlNjqAiIiIiIiIXJuKm4iI\niIiISCen4iYiIiIiItLJqbiJiIiIiIh0cipuIiIiIiIinZyKm4iIiIiISCen4ibShgYMGEBBQYHR\nMURERDrMAw88wF133WV0DJEuT8VNRERERL6RI0eO4OnpSUhICHv27DE6jkiXpuIm0gHq6+v51a9+\nxZQpU5g6dSq//e1vaW5uBuDdd99l6tSppKSkcN9993H06NFrbhcREeksli5dSkpKCjNmzGDZsmWt\n25ctW8aUKVOYMmUKP/3pT2loaPja7Tt37mTy5Mmtt/3y9Zdffplf/OIX3Hffffz1r3+lpaWF3/zm\nN0yZMoUJEybw05/+lMbGRgDKysp49NFHmThxInfeeSdbt25l06ZNzJgx47LM99xzD+vXr2/vl0ak\nzVmNDiDSHbz99tsUFBSwatUqmpqamDNnDitXrmTixIm8+OKLbNy4EQ8PDz7++GM2bdpEcHDwVbf3\n69fP6KciIiICQHNzM+vWrWPevHlYLBYWLVpEQ0MDRUVF/O53v2PZsmUEBATwwx/+kL/97W+kpKRc\ndXtsbOw1H2fz5s0sX74cX19f1q5dS3p6OitXrqSlpYVZs2axevVqZs6cyaJFi4iKiuLPf/4zBw8e\n5Hvf+x5btmyhuLiYnJwcoqOjOXfuHLm5uYwbN66DXiWRtqPiJtIBNm3axMMPP4zVasVqtXLnnXey\nbds2pk2bhslkYsmSJcyYMYOpU6cC0NjYeNXtIiIincXWrVuJjY3Fw8MDgKSkJDZu3EhFRQVDhgwh\nMDAQgEWLFmGxWPjwww+vuj0jI+OajxMfH4+vry8AU6ZM4fbbb8dmswEQGxvLmTNngIsF74033gBg\n0KBBbNiwAScnJ6ZMmcKqVauIjo5m/fr1TJw4EScnp7Z/QUTamQ6VFOkAZWVleHt7t1739vamtLQU\nm83GX//6VzIzM5kyZQr/8i//wuHDh792u4iISGeRmprKpk2bSExMJDExkU8++YSlS5dSXl6Ol5dX\n637Ozs5Yrdav3X49X/79WVZWxoIFC5gyZQopKSls2LABu90OQEVFBZ6enq37fl4op0+fzqpVqwBY\nv34906ZNu7UnLmIQFTeRDuDn50dFRUXr9YqKCvz8/ICLnwq+9NJL7NixgzFjxvDMM89cc7uIiIjR\nKisr2bVrFzt37iQ9PZ309HR2797Nvn37MJvNlJeXt+5bXV1NSUkJPj4+V91usVhaz/sGqKqq+trH\n/eMf/4jVauWjjz5izZo1jB8/vvV7PXr0uOz+z549S2NjI8OHD6epqYmNGzdy9OhRRo0a1VYvg0iH\nUnET6QC33XYbS5Ysobm5mdraWpYvX8748eM5fPgwTzzxBA0NDTg5OTF48GBMJtPXbhcREekMVq1a\nRXJy8mWHHFqtVsaMGUNDQwOZmZmcPXsWu93OM888w5IlSxg/fvxVt/v7+1NcXExpaSnNzc189NFH\nX/u4paWl9O/fHycnJ3JyctizZw+1tbUATJgwgaVLlwJw7Ngx7rnnHpqbmzGbzUybNo1nn32WCRMm\ntB5mKeJodI6bSBv79re/jcViab3+n//5n3z729/mzJkzTJ8+HZPJREpKSut5a6GhocyYMQObzYa7\nuzu/+tWv6N+//1W3i4iIdAbLli1j7ty5V2yfPHkyr776Kv/xH//B3LlzsVgsxMbG8r3vfQ9nZ+ev\n3X7vvfdy9913ExISwsyZMzl06NBVH/fhhx9mwYIFpKamkpiYyIIFC/j5z39OXFwcP/3pT1mwYAET\nJkzA3d2dF154ARcXF+Di4ZJvvfWWDpMUh2ayf35gsIiIiIhIF1RSUsKsWbPYtGnTZR+uijgSHSop\nIiIiIl3aSy+9xIMPPqjSJg5NxU1EREREuqSSkhImTpxISUkJDz/8sNFxRG6JDpUUERERERHp5DTi\nJiIiIiIi0smpuImIiIiIiHRynWY5gOLi87d8Hz4+bpSX17ZBmvbnSFnBsfIqa/twpKzgWHm7W1Z/\nf882StM9tMXvR+h+f886irK2H0fKq6ztw5GyQvv/juxSI25Wq+PMFORIWcGx8ipr+3CkrOBYeZVV\nOoIj/eyUtX04UlZwrLzK2j4cKSu0f94uVdxERERERES6IhU3ERERERGRTk7FTUREREREpJNTcRMR\nEREREenkVNxEREREREQ6ORU3ERERERGRTk7FTUREREREpJNTcRMRETZt2nBD+/3Xf/0X587ltXMa\nERER+SoVNxGRbi4//xzr16+9oX1//vOfExLSq50TiYiIyFdZjQ7QVk4VVJGTV0V0Ly+jo4iIOJQ/\n/OF3HDp0gLFjh3PHHVPJzz/Hn/70Ks8//x8UFxdRV1fHww//G6NHj+Xb3/42jz/+EzZu3EBNTTW5\nuafJyzvLE0/MZ+TI0UY/FRERkQ5XXdfIzoOFzBjft10fp8sUt7W7zrDzYCH3396XlBHhRscREXEY\nDz74bVJTPyAyMorc3FO8+ur/UV5eRlJSMlOnziAv7yy//OXPGD167GW3Kyoq5IUXXiItbTvLl3+o\n4nYVNTU1LFiwgMrKShobG5k3bx61tbX85S9/wWazERgYyPPPP4+Tk5PRUUVE5BvIK6nh5SV7Kaqo\no2+ELxF+bu32WF2muN07vg/H8ir5YOMx3FysjIsPMTqSiMhN++DTY+zOKWrT+xweHcD9E27sU8CB\nA2MA8PT04tChA6xYkYrJZKaqqvKKfePiEgAICAigurq67QJ3IUuXLiUyMpL58+dTWFjI3Llzqamp\nYfXq1Xh6evLLX/6SdevWMX36dKOjiojITco6VsLrKw5woaGZGaN6M3RAAKWl7ff7sMuc4+bn7cqz\nPxiFh6uNtz/OYdehQqMjiYg4HJvNBsC6dWuoqqrif/7n/3juuReuuq/FYmm9bLfbOySfo/Hx8aGi\nogKAqqoqfHx86NGjB1VVVZdtExERx2G321mddpqXl+ylpcXOozNjuGdcH8xmU7s+bpcZcQMIC/Tk\nJ7Pj+e+/7+GNjw7i6mwltk9Po2OJiNyw+yf0veHRsbZiNptpbm6+bFtFRQXBwSGYzWY2b/6UxsbG\nDs3UVUyfPp3U1FQmT55MVVUVr732GvX19cyaNQtPT08GDRrEqFGjrnkfPj5uWK2Wa+5zo/z9Pdvk\nfjqCsrYPR8oKjpVXWdtHZ8ta39jMKx9ksSnzLH7eLvz8eyPoG9aj9fvtmbdLFTeA3kFe/Oi+OP7w\nQTb/k7qPn8xOoP+XXkwREblcREQkhw/nEBwcQo8eF/+/vO22CfzsZz/h4MH9TJ9+FwEBAbz11hsG\nJ3U8y5cvJyQkhDfffJOcnByefvppmpqaWLJkCWFhYfz4xz9mw4YNTJw48Wvvo7y8tk2y+Pt7Ulx8\nvk3uq70pa/twpKzgWHmVtX10tqzl5+t5JXUvJ/PPExXixeP3xOLtYmnN2BZ5r1X8ulxxAxgQ7sNj\ndw/mldR9vLgkm//34FAigjpXWxcR6Sx8fHxITV112bbg4BDefvv91ut33DEV+OKXUp8+X4wK9unT\nl1deeb1jwjqYzMxMxowZA0B0dDTHjx8nIiKC8PCLk2iNHDmS/fv3X7O4iYiI8U6cq+Ll1L1UVjcw\nenAQ30mJxmbt2LPOusw5bl8V39eP788YxIX6Zv7wQRb5pTVGRxIRkW4mIiKC7OxsAPLy8ggKCqKy\nspKysjIA9u3bR0REhJERRUTkOnYcKOC372VSVdPA7Al9eXj6wA4vbdBFR9w+N2JQIHX1Tfxt7WEW\nLc7i6YeG0dPbxehYIiLSTcyePZuFCxcyZ84cmpqa+M1vfkNNTQ2PPvooTk5OhIaGakZJEZFOqqXF\nzoefHefjtFxcnS08flc8cVHGzZ/RpYsbwG1DelFb38SSTcd5YXEWTz80FC93rZcjIiLtz93dnRdf\nfPGK7ZMmTTIgjYiI3Ki6+iZeX3GA7OOlBPq48sR9cQT3dDc0U5c9VPLLpiVHMDU5nMKyWv6wOIva\nC5odTURERERErlRUXst/vZNB9vFSYnr78Iu5iYaXNugmxQ3gvvFR3JYQQm5RNX9aspf6xubr30hE\nRERERLqNQ6fKePbtdM6V1DA5MYwf3x+Pu4vN6FhANypuJpOJOXcMIGlgAMfOVvI/S/fR1NxidCwR\nERERETGY3W5nQ8ZZFi3O5kJDM9+bGs2Dk/phMXeeutR5knQAs9nE92cMIi6qJ/tPlPH6RwdpabEb\nHUtExCHcd9+d1NRohl4REelamppbeGftYd5bdwR3Vys/fXAIY+NDjI51hW5V3ACsFjP/fvdg+of1\nID2niL+tzcFuV3kTEREREeluztc2sOj9LDZlnSM8wINfzR1O/7AeRse6qi4/q+TVONss/Oi+OP77\n73v4LDsfN2cb37o9CpPJZHQ0EZEO9/DDD/Hcc4sICgqioCCfp5+ej79/AHV1dVy4cIEnn/wpgwYN\nNjqmiIhImzpTVM3LH+6lpPICiQP8eWT6IJydLEbH+lrdbsTtc67OVp6cHU9wTzfW7Mpl1Y7TRkcS\nETHEuHG3s23bZwBs2bKZceNuZ8aMu3n55dd49NHHee+9tw1OKCIi0rYyjxTz3DsZlFReYOaYSB69\ne3CnLm3QTUfcPufl5sT82Qk8/24mqZ+dwM3FyoShoUbHEpFuLPXYSvYU7WvT+xwSEMs9fWd87ffH\njbudV175E/feez9bt27m8cef5P333+Ef/3iHxsZGXFxc2jSPiIiIUex2Oyu3n2LplpM42cw8dvdg\nEqMDjI51Q7rtiNvnfL1ceOqBBLzcnXj3kyPsOFBgdCQRkQ7Vp08UpaXFFBYWcP78ebZs2YSfXwD/\n+79v8tRTPzM6noiISJuob2zmtRUHWLrlJD29nFk4Z5jDlDbo5iNunwv0deMn98fz33/fw5srD+Hq\nZCWhn5/RsUSkG7qn74xrjo61l5Ejx/D6668ydux4KirKiYrqB8DmzRtpamrq8DwiIiJtqazqAi9/\nuI/ThefpG+rN47Ni8XJ3MjrWTen2I26fCw/05MffisdqNfHqsv3knC43OpKISIcZP/521q9fy223\nTSQlZTqLF7/Hk0/OIyZmMKWlpaxatcLoiCIiIt/IsbxK/uPtdE4XnmdsXDD/78EhDlfaQCNul+kb\n6s0P74njT//M5sUP9/L/HhxCZLCX0bFERNrdwIExbN68s/X6e+8tab08Zsx4AKZPvwt3d3dqa893\neD4REZFvYtu+fN5ek0Nzi50HJ/Vj0rBQh51JXiNuXxET6csP7oqhobGZPyzOIq+42uhIIiIiIiJy\nE1pa7Cz+9ChvrjqEk9XCT+5PYHJimMOWNlBxu6rE6AC+mxJNzYUmFi3OoriizuhIIiIiIiJyA2ov\nNPGnJdms3XWGIF83fjE3kZhIX6Nj3TIVt68xNj6EByb0paK6gRfe30NFdb3RkURERERE5BoKymr5\nz7+ls/9EGbF9evKL7yQS5OtmdKw2oeJ2DXckhXPX6N4UV1xg0eIsqusajY4kIiIiIiJXsf9kKf/5\ndjoFZbWkJIXzo/vicHPpOlN6qLhdx8wxkUwcFkpecQ1/+mc2Fxo0LbaIiIiISGdht9tZt/sMf/wg\nm4amZh6ZPpD7J/TFbHbc89muRsXtOkwmEw9O6seowUGcOFfFyx/uo7Gp2ehYIiIiIiLdXmNTC3/9\nOId/bDiKp5sTC/5lKKNjg42O1S5U3G6A2WTie9OiGdLPj0Ony/nz8gM0t7QYHUtEREREpNuqqmng\n9+/vYcvefCKCPPnV3ESienkbHavdqLjdIIvZzKMzYxgY4cOeoyW8tTqHFrvd6FgiIiIiIt1ObuF5\n/uPt3Rw7W0nSwAB+9tBQfL1cjI7VrlTcboLNauHxe2KJDPZi+/4C/rH+KHaVNxERERGRDpOeU8Rz\n72ZQVlXPrHF9+MFdMTjbLEbHancqbjfJ1dnKk/fH08vfnQ0ZZ1m+9aTRkUREREREurwWu51lW07w\n6rL9mDDxw3tiuXNUb4deVPtm3ND8mM899xzZ2dmYTCYWLlxIXFxc6/cmTJhAUFAQFsvFlvvCCy8Q\nGBh4zds4Og9XG/NnJ/D8uxms2HYKNxcbdwwPMzqWiIiIiEiXVN/QzP+tOkjG4WL8vF144t44QgM8\njI7Voa5b3Hbt2sXp06dZvHgxx48fZ+HChSxevPiyfd544w3c3d1v6jaOroeHM089MITn3s3g/Q1H\ncXW2MDYuxOhYIiIiIiJdSkllHS9/uI8zRdUMCOvBY7MG4+nmZHSsDnfdQyV37NjBpEmTAIiKiqKy\nspLq6uo2v40j8u/hylOzE3B3sfLXj3NIzykyOpKIiIiISJdx5EwFz76dzpmiam5LCGH+AwndsrTB\nDYy4lZSUEBMT03rd19eX4uJiPDy+GJp85plnyMvLY9iwYcyfP/+GbtNV9PL34CezE/jvf+zh9Y8O\n4OpsJSbS1+hYIiLSCdTU1LBgwQIqKytpbGxk3rx5vP76663fLyoqYtasWTz66KMGphQR6Zw+2Xma\nV5dkY7fDnDv6M2FoqNGRDHVD57h92VdnUXziiScYO3Ys3t7ezJs3j7Vr1173Nlfj4+OG1Xrrs8H4\n+3ve8n18k8f8lZsTv34jjVeW7uM/fzCK6N7XL29GZL0VjpRXWduHI2UFx8qrrF3T0qVLiYyMZP78\n+RQWFjJ37lzWrFnT+v3vf//7zJw508CEIiKdT3NLC4s/Pcb69LO4u1h57O7BDLyB99Zd3XWLW0BA\nACUlJa3Xi4qK8Pf3b71+9913t14eN24cR44cue5trqa8vPamgl+Nv78nxcXnb/l+volgbxf+feZg\nXkndxzOv72DBQ0MJu8YJk0Zm/SYcKa+ytg9HygqOlbe7Ze1Oxc/Hx4fDhw8DUFVVhY+PT+v3tm/f\nTu/evQkODjYqnohIp1NzoZE/L9vPgVPlhAV6Mu/uGAJ83IyO1Slc9xy30aNHt46iHThwgICAgNZD\nHs+fP88jjzxCQ0MDALt376Zfv37XvE1XltDPj0emD6S2volFi7MoLLv1MioiIo5r+vTpnDt3jsmT\nJzNnzhwWLFjQ+r2//e1vfOc73zEwnYhI55JfWsOzb6dz4FQ58VE9eeGJsSptX3LdEbehQ4cSExPD\nAw88gMlk4plnniE1NRVPT08mT57MuHHjmD17Ns7OzgwaNIiUlBRMJtMVt+kuRg4Oora+iffWHeGF\n9/fw9JxhXX4VdxERubrly5cTEhLCm2++SU5ODgsXLiQ1NZXCwkJqa2sJDw+/7n201akE4Fijncra\nPhwpKzhWXmW9NemHCvn9uxnUXmjivgn9mDN1IBazCTcXm9HRbkp7vrY3dI7bU089ddn16Ojo1stz\n585l7ty5171NdzJxWCi19U0s/ewEixZnseChoXh109lvRES6s8zMTMaMGQNc/N1ZVFREc3Mzmzdv\nJjk5+Ybuoy1OJYDud0huR1HW9uNIeZX1m7Pb7azddYZ/bjqGxWzmX+8cxMiYIMpKqztd1utp79MJ\nrnuopHwzM0ZGMCUpjPzSWv64OJvaC01GRxIRkQ4WERFBdnY2AHl5ebi7u2OxWNi3b99lH4KKiHRH\njU3N/GXVIT7YeAwvdyeenjOUkTFBRsfqtFTc2onJZOL+2/syNi6Y04XneenDvTQ0NhsdS0REOtDs\n2bPJy8tjzpw5zJ8/n1//+tcAFBcX07NnT2PDiYgYqKK6nv/++x627S8gMtiTX80dTmSwl9GxOrWb\nXg5AbpzJZGJuSjR1Dc2k5xTx6rL9PH5PLFaL+rKISHfg7u7Oiy++eMX2P//5zwakERHpHE4VVPHy\nh/soP19Pckwg302JxsnWNufydmUqbu3MbDbxb3cO4kJDE3uPl/J/Kw/yb3fGXP+GIiIiIiJdzK5D\nhfxl1SEam1r41m1RpIwIx2QyGR3LIWjopwNYLWbmzYqlb6g3uw4V8e4nh29oUXIRERERka6gxW4n\n9bPj/Hn5AcxmEz+8L46pyREqbTdBxa2DONss/Pi+OMIDPNiUdY6/rT5kdCQRERERkXZXV9/E/6Tu\nY+X20/j3cOHn3x5GQl8/o2M5HBW3DuTmYuMnsxMI9HVjyadHWZ122uhIIiIiIiLtpriijufezWDP\n0RIGRvjwy7nD6eXvYXQsh6Ti1sG83J14anYCfj1cWbLpOJv25BkdSURERESkzR3OLefZt9PJK65h\n4tBQnrw/Hg9Xx1pQuzNRcTNAT28Xnv3BSDzdbLyz9jA7DxYaHUlEREREpM1s2pPHC+9nUVffxHdS\nBvDQHf01s/ot0qtnkNAAT35yfwIuzhb+b+VBso+VGB1JREREROSWNDW38O4nh/nb2sO4Olt56oEE\nbkvoZXSsLkHFzUARQZ786L54LGYTry7bz+HccqMjiYiIiIh8I9V1jfxhcRafZuYR6u/OL+cmMiDc\nx+hYXYaKm8H6h/XgsVmxtLTYeXHJXk4VVBkdSURERETkpuQVV/Ps27vJya1gSD8/np4zDP8erkbH\n6lJU3DqBuKie/Oudg6hvaOYPi7PJL60xOpKIiIiIyA3JOlbCf72TQXHFBWaM6s28e2JxdbYaHavL\nUXHrJJIGBvKdlAFU1zXywvtZlFTWGR1JRERERORr2e12Vqed5uUle2lpsfPozBjuGdcHsxbVbhcq\nbp3I+IRe3H97X8rP1/PC+1lU1jQYHUlERERE5AoNjc288dFBlmw6Tg9PZ342ZyhJAwONjtWldZni\nVlhTxM6ze2hoduyykzIinOkjIygqr2PR+1nUXGg0OpKIiIiISKvy8/X87u+ZpB0sJCrEi1/NTaR3\nkJfRsbq8LnPw6cqTn5BZtBcXiwvDAuNIDk4k0isCkwMO1d4zrg+19U1szMzjT//M5qnZQ3B2shgd\nS0RERES6uRPnqng5dS+V1Q2MHhzEd1IGYLPqfWpH6DLF7b5+M4noGcLGEzvYdm4X287tIsDNj+Sg\nREYED6OHs7fREW+YyWTiocn9qbvQRNrBQl5J3csT98Vjs3aZAVIRERERcTA7DhTw1uocmltamD2h\nL3cMD3PIQRJH1WWKm7ezJw/GzWRC0G0cLjtGWkE62cX7WXFiDR+dWEu0bz9GBicS5xeDzWIzOu51\nmU0mHp4+kAsNzWQdK+H1jw7w6MwYLGaVNxERERHpOC0tdj787Dgfp+Xi6mzh8bviiYvqaXSsbqfL\nFLfPmU1mBvbsz8Ce/altrCOjKJu0/HQOlR3hUNkRXK2uJAYmkBw8jAjPzv0pgdVi5tGZMfzxg2wy\nDhfz9seH+e60aM3UIyIiIiIdoq6+iddWHGDv8VICfVx54r44gnu6Gx2rW+pyxe3L3GyujO2VzNhe\nyRTUFJKWn8Guggy25O1gS94OgtwDSQ4aRlLQMLydPY2Oe1VONgtP3BfH7/+xh6378nFzsTJ7Qt9O\nXThFRERExPEVltfy0pK95JfWEtPbh0fvHoy7S+c/cq2r6tLF7cuC3AO5u+807uwzhUNlR0gryGBf\n8QGWHV/NihNrGOQ7gOTgRGL9BmI1d66XxdXZypP3x/O7v+/hk91ncHOxctfoSKNjiYiIiEgXdehU\nGa8u20/NhSYmJ4Zx/4QonbJjsM7VUDqAxWxhsN9ABvsNpLqxhozCbNLyd7O/9BD7Sw/hbnMjMXAI\nI4MTCfUI6TQjW55uTsyfncDz72awbMtJ3JytTEoMMzqWiIiIiHQhdrudTzPz+Mf6o5hM8L2p0YyN\nDzE6ltANi9uXedjcGR86ivGho8irzictP53dBXvYfHYbm89uo5dHMMnBiQwPHIKnk4fRcfHxdOap\nBxJ4/t1M/r7+KK7OVkbHBhsdS0RERES6gKbmFt5bd4TNWefwdLMxb1Ys/cN6GB1LLunWxe3LenkE\nc2+/O7k7ahoHSnMuHkpZcpAPj37E0mOrGNxzIMnBiQzuGY3FbNxaFQE+bsyfncDv/p7JW6tzcHO2\nMqS/v2F5RERERMTxVdU28OrS/Rw5U0FYgAdP3BtHT28Xo2PJl6i4fYXFbCHOP4Y4/xjON1STXpjF\njvzd7C05wN6SA3jY3EkKGkpycCK9PIwZ7QoN8ODH34rnhfez+N/l+/nxt+IZ1NvXkCwiIiIi4tjO\nFFXz8od7Kam8wLAB/nx/+iCcnbSodmej4nYNnk4e3B42htvDxnDm/DnS8nezu3APn57ZwqdnthDm\n2YvkoEQSgxLwsHXstKhRvbz54b2x/Omf2bz84T6eejCBqBDHWWRcRERERIyXeaSYNz46SH1jMzPH\nRHLn6N5aeqqTUnG7QWGeIYR5zmRW3+nsLzlEWkE6B0oP88+jy0k9tpJYv0GMDE5koG//DjuUclBv\nX35w12BeXbaPP32QzYKHhhLqb/y5eCIiIiLSudntdlZuP8XSLSdxspl57O7BJEYHGB1LrkHF7SZZ\nzVYSAmJJCIilsv48uwsz2ZGfTlbxPrKK9+Hl5Nl6KGWwe2C75xk2wJ+Hpw3kzVWHWPR+Fk/PGUqA\nj1u7P66IiFxfTU0NCxYsoLKyksbGRubNm0dCQgJPPvkklZWVBAYG8oc//AEnJyejo4pIN1Lf2Mxf\nVh1id04RPb2c+eG9cYQHds41jeULKm63wNvZk0nh45kYNo7c82cvzkpZmMX63M2sz91MhFcYI4MT\nGRYQj5ut/crU6Nhgauub+Mf6o7zwfhZPzxmGj6dzuz2eiIjcmKVLlxIZGcn8+fMpLCxk7ty5TJgw\ngTFjxvDd736XV155hZycHOLi4oyOKiLdRHF5Hb99N5PThefpG+rN47Ni8XLXh0eOQMWtDZhMJiK8\nwojwCuOevjPYW3KQtPx0DpW5JNDSAAAgAElEQVQd4XTVGZYc/Yh4vxiSgxOJ9u2H2dT2ixdOTgyj\n9kITy7eeZNHiLH720FA8XLWyvYiIkXx8fDh8+DAAVVVV+Pj4sHHjRt59910AHn/8cSPjiUg3c+hU\nGW+sOkTF+XrGxgUz544B2KxaVNtRqLi1MZvFxrDAeIYFxlNRX8mu/EzSCtLJKMomoyibHs7eJAUN\nZZrLeGy07SjcXaN7U3uhiXXpZ/jjB1k89cAQXJ31IxaRb8Zut1NeX0F+TSH5NYUU1hQxpHoQgzxi\njI7mMKZPn05qaiqTJ0+mqqqK1157jX/913/lH//4B9u3b6dv37784he/0KGSItJuWux29hwpYc2u\n0xzPq8Jsggcn9WPSsFBMmoTEoZjsdrvd6BAAxcXnb/k+/P092+R+2prdbudkVS5p+elkFGZzofkC\nAH28I0gOTmRoQDyu1rZZJ6PFbuet1YfYtq+A6PAePHl/PDbrrU+W0llf26tR1vbhSFnBsfIanbXF\n3kL5hS8KWkFN0cWvtYXUNzdctu/wXvF8d8BDt/R4/v7d5zyK5cuXk56ezrPPPktOTg4LFy7k2LFj\nvP322wwZMoRf/OIXDBw4kIce+vrXtKmpGWsb/D8uIt1LQ2Mzn6afYdnmY+QV1wCQODCQ2ZP7Ex2h\nZaQckYZjOoDJZKKPdwR9vCO4r99dZBfvJ7M0i32FOZyoPM0/j6wgwT+W5OBh9PeJuqVDKc0mE9+d\nGs2F+mYyjhTzv8sO8NiswVgtGgYX6e5a7C2U1pVTUFv4pZJ2sag1tDRetq/VZCHAzZ9g90CC3AMI\ncg8kxD2QmPA+lJbWGPQMHE9mZiZjxowBIDo6mqKiIoKCghgyZAgAo0ePZufOnde8j/Ly2jbJYvQH\nBDdDWduHI2UFx8rbmbJW1zWyMfMsGzLOUlXbiMVsYkxsMFNGhNPLz71TZb0eR8oKbZP3Wh9uqrh1\nMCeLjeFBQ5gWO47DZ3LZVZB5aVKTTHYXZuLj3IPk4GEkByfi59rzGz2GxWzm3+6K4aUl2WQdK+Gt\n1Yd4ZMYgrckh0k202FsoqSslv6aIgi8XtNpiGr9a0MxWAj8vaG6BBHsEEuwWgJ9rz6subWI260Og\nmxEREUF2djZTpkwhLy8Pd3d3RowYQVpaGsnJyRw4cIDIyEijY4pIF1BSUcfa3WfYsvccDY0tuDpb\nmZoczqRhYZq0rotQcTOQr4sPKb0nMiViAscrT5GWn05mUTYfn9rAx6c20LdHJMnBwxniH4uL9eb+\nwdmsZubdE8ui97PYcaAQV2crD03ur2OZRbqQ5pbmiwWttoj86sLWkbTC2mKaWpou29dmthLkdnHk\n7OIoWiDB7gH0dPHtsLUnu6PZs2ezcOFC5syZQ1NTE7/+9a8ZMGAATz31FC+99BJ+fn489thjRscU\nEQd2uuA8H+88TXpOMS12Oz6ezswaG8a4+BDNddDF6KfZCZhMJvr2iKRvj0i+1X8mWUX72JG/m6MV\nJzhWcZIPjixjqH8cycHD6Nujzw2XLxcnKz++P57fvZfJp5l5uLlYuWdcVDs/GxFpa80tzRTXlZBf\nU0R+TUHrOWhFtcU02Zsv29fJbCOktZgFto6k9XT1aZcZbeXa3N3defHFF6/Y/pe//MWANCLSVdjt\ndvafLGPNzlwOnS4HINTfnZQR4SQNDNQpMl2Uilsn42xxYkTwMEYED6Okroyd+ensLMggrSCdtIJ0\n/Fx8L34/KJGerj7XvT93FxvzZyfw/HuZrNx+GjdnGykjwjvgmYjIzWpqaaKotqT10Mb82i8KWou9\n5bJ9nSxO9PIIaT0H7fNRNF+XHipoIiJdVFNzCzsPFrJ2Vy5nL004MjDCh6kjwomJ9NWRVV2cilsn\n5ufqy/Q+dzA1chLHKk6Qlp/BnqK9rDq5jlUn19Hfpy8jgxNJ8B+Mk+Xrp5L29nDmqUvl7YONx3Bz\nsTIuPqQDn4mIfFljSxNFtcWtBa3sSBmnyvIoriu5oqC5WJyJ8Awl6EsFLdg9kB7O3ipoIiLdRF19\nE5uzzrEu/Qzl5+sxm0yMGBRISlI4EUHdZ6be7k7FzQGYTWb6+/Slv09f7u8/k8yifaTl7+ZI+TGO\nlB9jscWZoQHxjAxJJNIr4qqftvj1cGX+7AR++14mb3+cg6uzleHRAQY8G5Huo7G5kYLa4kszN16a\nybG2kOLaUuxcvhKLq9WF3l5hFycIcQ8g2D2IIPcAejh76xNUEZFuqvx8PevTz7ApK4+6+macbRYm\nJYZyR2IYfj1cjY4nHUzFzcG4WF0YFTKcUSHDKaotYWd+OmkFGWzP38X2/F0EuPmRHJRIUtBQfFx6\nXHbbED93fjI7nv/++x5eX3EAFycLsX2+2cyVIvKFhuYGCmqLWs89+3wkraSu7IqC5mZ1pY93xGXn\noMWE96HpvFkFTUREAMgrrmbNrlzSDhTS3GLHy81GyrgIbh/SCw9Xm9HxxCAqbg4swM2PO6NSmN7n\nDg6XHyMtP53s4v2sOLGGj06sJdq3H8nBicT7xWCzXPxH3jvIix/dF8cfPsjmf1L38ZPZCfQP63Gd\nR5Kb1WJvobaxjurGmi/+NFRT3VhLdWM11Q0Xv5qtYG6x4mJxxtnqjIvF+bLLzhZnXKyXvn7lus1s\n1Rv9DnahqZ7CqxS00gvlVxQ0d5sbUT16f1HQ3C6eg+bl5HHFz83X1ZPiasdZp0ZERNqe3W7ncG4F\na3blsvd4KQCBvm6kJIUxanAQNqtmAO7uVNy6ALPJzEDf/gz07U9tYx2ZRdmk5adzqOwIh8qO4Gp1\nZVhgPCODE4nwDGNAuA+P3T2YV1L38eKSvSz4lyGEB+r46GtpaG64VL5qvlLGrrxc01hDTWPtFW/k\n25rZZP6i0N1Q6XP6Yr8vb7+0zWrWfwefu9B0gYJLU+zn115coPrzgvZVnjYP+vaI/GIGx0tfPZ08\nDEguIiKOpqXFTsaRYtbsPM3J/Isf4vUN9WZqUjjx/fy0Dq+00ju1LsbN5sqYXsmM6ZVMQU0Rafnp\n7CrIYGteGlvz0ghyDyQ5aBhJYUN5ZMZA3lhxkEWLs3h6zjCCfN2Mjt8hbnQ0rKaxhvMNF4tYw1cW\nLb4aEybcbW542NwJdAvA08kdd5s7njZ33J3c8bBd+vOly0EBPThbUEJ9cwMXmuu50FRPfXM9F5rr\nqW+6/Otl2y79+Xz/8w3nKWkquWJq+JthNVmuOdLXw9MDe4P58nL4lf0/v+xscXKItcHqmuq+Mnp2\n8XJ5fcUV+3o5edLfp++lghZw6Vy0QDyc3A1ILiIijq6+sZmte/P5ZHcuxRUXMAFD+vkxdUQEfUO9\njY4nnZCKWxcW5B7A3X2ncWefKeSUHyUtP529xQdYdnw1K06sYZBvf267vQ8bN9Xzwvt7ePqhYfT0\ndjE69k376miYqaaZ/LLSK0bDaho/HxG7sdEwJ7MNDycPgtwD8LB54G5zx8PJDQ+bBx42NzycPL4o\nYzZ33GyuNz3Ln8Vswc3mhputbUpzU0vTFQXvQtNXS1/DZaXvauWwvL6S+tr6y2c4LLi5LDazDWeL\n01WL3rVGBj+/7HzZZadbmkGxtrGW/EujZp+PoOXXFFJRX3nFvt5OXkT79GudZv/zETT3NvoZiYhI\n91ZV28CnGWf5NDOP6rpGrBYz4xNCmJIU3m0+RJdvRsWtG7CYLcT0jCamZzQ1jbVkFGaxIz+d/aU5\nQA5ew505XxjI75ZV8fN7J+Dt4WxY1s40GnatJRY6K6vZiofZioft1keB7HY7jS1NrSXPzctKfknZ\ndUvf1UpidV059c31t3T4qNOlEnjNw0EvbbeYLFSfqeJEyRkKagqpbLjy/LEezt4M9O3/pXXQgghy\nC8DNplm6RESk7RWW1/LJrjNs3ZdPY1ML7i5WZozqzcRhoXi7O957Dul4Km7djLvNjXGhoxgXOopz\n1QWXDqXMpDEwlxpy+eXWTKb1H83o0MQ2OUfnRs8N+yajYe42d4LcAy6OhNk8WkfDgn19sddbb3k0\nrLszmUw4WWw4WWx4Onng7+OJe9M3P3TDbrfT0NJ4qfRduObhoJ8fOvp1JbGioYqG5oYbelxfFx8G\n9RxAsNsX56AFuQfganW80WUREXE8x89VsmZnLpmHi7EDft4u3DE8jLFxITg7df7TCqTzUHHrxkI8\ngrin3wxmRk3lQGkOH2Rvosyay0enVrPq9BoG9xxIcvAwBvccCDjOaJi/vyfFxZqhr7MxmUwXJ0ix\nOAG3PhlOi73lisM+P7/c2NJE3+BQnBvccVFBExGRDtZit7P3WClrdp7myNmLh+VHBHqSMiKcxGh/\nLGZ9mCw3T8VNsJgtxPnHMHjCIF5dmUF26V48ehWyt+QAe0sO4Gp1xWqxUF1fc8ujYW11bpiI2WTG\n1epyceTsKkf3+vdUgRcRkY7V2NRC2oEC1uzKJb+0FoDBfXyZmhROdISPlvGRW6LiJq3MZhOPTh/K\nyx/a2JdZyuBBVsIGlLO/9BDOTk4EuPi3joZ9dQTsy9cd8dwwERERkW+quq6RVTtOsT7jLJXVDVjM\nJkYNDiIlKZzQAC0PI21DxU0uY7WYeWzWYP64OIv9ByvxtfXn1yl3ERDgpdELERERkS8prbzAuvQz\nbNl7jrr6ZlycLExJCmNyYhi+XjpUX9qWiptcwdlm4Yn74vn9P/bwWXY+bi42HvtWgtGxRERERDqF\n3MLzrNmVy+5DRTS32PH1cmHGyN6MT+iFm4veXkv7uKG/Wc899xzZ2dmYTCYWLlxIXFzcFfssWrSI\nrKws3nnnHXbu3MmPfvQj+vXrB0D//v355S9/2bbJpV25uVh5cnY8v3svkzU7cyk738DUpDAigm59\nUgkRERERR2O32zl4upw1O3M5cLIMgF5+7kxJCmfG+L5UlNcYnFC6uusWt127dnH69GkWL17M8ePH\nWbhwIYsXL75sn2PHjrF7925sNlvrtqSkJF566aW2TywdxsvNifmzE3htxQF2HSxg18EChvTzY+aY\nSMIDVeBERESk62tuaWH3oSLW7Mwlt6gagOjwHqSMCCe2T09MJhM2qyZZk/Z33eK2Y8cOJk2aBEBU\nVBSVlZVUV1fj4fHFiZa//e1vefLJJ3nllVfaL6kYwtfLhZ89NJRzFRd4e+VB9hwtYc/REhU4ERER\n6dIuNDTxWXY+63bnUlpVj8kEidEBTB0RTmSwl9HxpBu6bnErKSkhJiam9bqvry/FxcWtxS01NZWk\npCR69ep12e2OHTvGo48+SmVlJY8//jijR4++5uP4+Lhhtd76IoT+/o5TJBwpa0CAF/FP+rPnSDF/\nX5vTWuBGxgbz4B0DiAz55gsztwdHem2Vtf04Ul5lFRHpHCqr61mfcZZNe/KoudCEk9XMhKG9uCMp\nnIAerkbHk27sps+etNu/WMeroqKC1NRU3nrrLQoLC1u39+7dm8cff5ypU6dy5swZvvOd7/DJJ5/g\n5PT108SXl9febJQrONLCy46UFS7mLSmpJszXlf/3QAIHTpaxbOtJduzLZ8e+fIYN8Gfm6MhOMeWt\nI722ytp+HClvd8uq4icinVF+aQ1rd+WyfX8BTc12PFxt3D0mktuH9sLTTUsdifGuW9wCAgIoKSlp\nvV5UVIS/vz8AaWlplJWV8dBDD9HQ0EBubi7PPfccCxcuZNq0aQCEh4fj5+dHYWEhYWFh7fQ0pCOZ\nTCYG9+lJTKQv+0+WsWzLSTIOF5NxuJjEAf7cNSaSUH/jC5yIiIjI9Rw9W8HHablkHbv4fjeghytT\nRoQzenAQTrZbPxpMpK1ct7iNHj2al19+mQceeIADBw4QEBDQephkSkoKKSkpAJw9e5ann36ahQsX\nsmLFCoqLi3nkkUcoLi6mtLSUwMDA9n0m0uFMJhOxfXoyONKXfSfKWL71BOmHi0k/XExidAAzR/em\nlwqciIiIdDItLXb2HC1hza7THM+rAqBPiBcpSeEM7e+P2WwyOKHIla5b3IYOHUpMTAwPPPAAJpOJ\nZ555htTUVDw9PZk8efJVbzNhwgSeeuopNmzYQGNjI7/+9a+veZikODaTyURcVE9i+/iy93gpy7ee\nJD2niIycIhKjA7hrTCS9/NyNjikiIiLdXENjM9v3F7B2Vy6F5XUAJPT1I2VEOP1CvTGZVNik87qh\nc9yeeuqpy65HR0dfsU9oaCjvvPMOAB4eHvz5z39ug3jiSEwmE/F9/YiL6kn2pQK3O6eI9Jwihg8M\n4M7RKnAiIiLS8arrGvk08ywbMs5yvrYRq8XE2LhgpiSFE6L3JuIgtLS7tDmTyURCXz/io3qSfexi\ngdt1qIjdh4pIGhTInaN66z9JERERaXfFFXV8susMW/ado6GxBVdnK9OSI5iUGEoPD2ej44ncFBU3\naTcmk4mEfn7E9+1J1rESlm89yc6Dhew6WMiIQYHcObo3wT1V4ESk66qpqWHBggVUVlbS2NjIvHnz\neP3116mtrcXNzQ2ABQsWMHjwYIOTinQtpwqqWLMzl905Rdjt4OvlzB1jwxgbH4Krs97+imPS31xp\ndyaTiSH9/Eno68eeoxcLXNrBQnYeuljg7hodSZCvm9ExRUTa3NKlS4mMjGT+/PkUFhYyd+5c/P39\nef755+nfv7/R8US6FLvdzr4TZazZeZqc3AoAwgI8SBkRzvDoAKwWs8EJRW6Nipt0GJPJxND+/iT0\n82PPkUsF7kAhOw8WkjwoiDtH91aBE5EuxcfHh8OHDwNQVVWFj4+PwYlEup6m5hZ2Hixkza5c8opr\nABjU24epIyIY1NtHE45Il6HiJh3ObDIxbIA/Q/r7sedIMcu3nmTHgQLSDhYwMiaIO0f1JlAFTkS6\ngOnTp5OamsrkyZOpqqritddeY9GiRbz00kuUl5cTFRXFwoULcXFxMTqqiMOpq29ic9Y51qWfofx8\nPWaTieRBgUxJCiciyNPoeCJtTsVNDHOxwAUwpL8/mYeLWb7tJNv3F5B2oJCRMYHMGN2bQB8VOBFx\nXMuXLyckJIQ333yTnJwcFi5cyL//+78zYMAAwsPDeeaZZ3jvvfd45JFHvvY+fHzcsFrbZhFgf3/H\neTOrrO3DkbLC1fOWVtax4rMTrEk7Re2FJlycLNw1rg8zx0YRYOAHv4702ipr+2nPvCpuYjizyURi\ndABDB/iTcbiYFVtPsm1/ATsOFDJy8MVZKANU4ETEAWVmZjJmzBjg4lI6RUVFTJgwAYvlYhGbMGEC\nq1evvuZ9lJfXtkkWf39PiovPt8l9tTdlbR+OlBWuzHu2uJq1O3NJO1hIc4sdb3cn7h3fh9uG9MLd\nxQbNzYY9P0d6bZW1/bRF3msVPxU36TTMJhPDowMYNsCf9JwiVmw7xbZ9BezYX8io2CBmjOpNQA9X\no2OKiNywiIgIsrOzmTJlCnl5ebi5ufHII4/w0ksv4eXlxc6dO+nXr5/RMUU6LbvdTk5uBWt25rLv\nRCkAwT3dmJIUzsiYIGxWTTgi3YeKm3Q6ZpOJpIGBJA4IIP1wEcu3nmTr3nx27C9g1OCLBc5fBU5E\nHMDs2bNZuHAhc+bMoampid/85jeUl5fz3e9+F1dXVwIDA/nhD39odEyRTqe5pYUtWXl8sO4wpwou\njmD0C/UmZUQ48X39MGvCEemGVNyk0zKbvyhwu3OKWLHtJFv25rN9fwGjY4OYMbI3fipwItKJubu7\n8+KLL16xfdq0aQakEen8Wlrs7DhQwIptJymuuIAJGNbfn5QR4UT18jY6noihVNyk0zObTYwYFMjw\n6AB2HSpkxbZTfJadz7Z9BYyODWbGqAj8vFXgREREHJXdbifzSDGpn50gv7QWq8XE1JG9GRcbpJmm\nRS5RcROHYTabSI4JImlgIDtbC9w5tu3LZ2xcMNNH9qant6bUFhERcRR2u50Dp8r4cPMJThecx2wy\nMS4+mLtGRzIgyt+hJqYQaW8qbuJwzGYTI2OCSBoYwK6DFw+h3JR1ji178xkbH8K3pw9CR76LiIh0\nbsfyKkndfJyc3AoAkgYGcPfYPgRphE3kqlTcxGFZzGZGDg4iaVAAOw9eHIHbtCePrXvPMTYuhOkj\nI/D10giciIhIZ3KmqJqln50g61gJAHFRPblnXB/CAx1rvS6RjqbiJg7PYjYzanAwIwYFknagkNVp\nuWzck8eWvecYGx/C9GQVOBEREaMVlteyfMtJdh4sxA70D/XmnvFR9A/rYXQ0EYeg4iZdhsVsZnRs\nMHeO78uKTcf4aPtJNmbmsSX7HOPiQ5g+sjc+ns5GxxQREelWys/X89GlmaGbW+yEB3pw7/goBkf6\nYtK0/iI3TMVNuhyLxcyYuGCSYwLZcaCAj7ad4tPMPD7Lzmd8QgjTkiNU4ERERNrZ+doGVqed5tPM\nPBqbWgjydWPWuD4MG+CvddhEvgEVN+myrBYzY+NCGBkTxI79BXy0/RQbMs6yOesctyWEMG1kBD08\nVOBERETaUl19E5/sPsPaXblcaGjG18uZmaMjGRUbhMVsNjqeiMNScZMuz2oxMzY+hJGDg9i+/+II\n3PqMs2zOPtc6AqcCJyIicmsam5r5NDOPVTtOU13XiKebjVlj+3DbkBBsVovR8UQcnoqbdBtWi5lx\n8SGMGhzEtn35rNx+ivXpn4/A9WJacjjeKnAiIiI3pbmlha1781mx7RTl5+txdbYwa2wkk4eH4eKk\nt5oibUX/mqTbsVrMjE/oxejYYLbuy2fV9lOsSz/D5qw8bhvSi6nJEXi7OxkdU0REpFNrsdvZfaiI\nZVtOUFheh5PVzNQR4UxNjsDD1WZ0PJEuR8VNui2rxcxtCb0YExvM1r35rNxxik92n2HTnjxuH9qL\nqSMi8FKBExERuYzdbmfv8VJSPzvBmaJqLGYTtw/pxYxRmr1ZpD2puEm3Z7WYuW3IpRG4vedYueM0\na3edYeOePCYMCSVlRLgKnIiICHA4t5wPPzvBsbOVmICRMYHMHNuHgB6uRkcT6fJU3EQusVnN3D40\nlDFxIWzZe45VO06zZlcun+45y4ShlwqcmwqciIh0P6cLzvPh5uPsP1kGwJB+fswa14dQfw+Dk4l0\nHypuIl9hs5qZMDSUsXHBfJadz6odp1izM5eNmXlMGNaLlKRwPFXgRESkG8gvrWHpZydIP1wMwMAI\nH+4Z34eoEG+Dk4l0PypuIl/DZrUwcVgo4+K/KHAfp+XyaUYeE4eFMiUpTAVORES6pJLKOlZsO8W2\nffnY7RAZ7MW94/swqLev0dFEui0VN5Hr+HKB25R1jtVpp1mddpoNmWeZNCyUKUnhmj1LRES6hKqa\nBlZuP8WmrDyamu308nNn1rg+DOnnh8lkMjqeSLem4iZyg2xWC5MTwxgfH8LmSwVu1Y7TrM9QgRMR\nEcdWe6GRNbtyWbf7LPWNzfh5u3D32EiSBwVhNquwiXQGKm4iN8nJZmHy8DDGJ4S0jsCt2nGaDRln\nmZQYxh3Dw1TgRETEIdQ3NrMh4ywfp52m5kIT3u5O3H97FGPjQ7BazEbHE5EvUXET+YacbBbuuFTg\nNu/JY/XOXFZuP8WGjDNMGhbGHUlhuLuowImISOfT1NzCZ9nn+GjbKSprGnB3sXLfbVFMHBaKs81i\ndDwRuQoVN5Fb5GyzcEdSOOOH9GJjZh5rdp7mo+2nWJ9xhsmXRuDcVOBERKQTaGmxk3awgGVbTlJS\neQFnm4UZo3qTkqTfVSKdnYqbSBtxtllIGRHO7UN6sXFPHh/vPM2KbadYl36WyYmhKnAiImIYu93O\njn35vL3yAHklNVgtJiYlhjJ9ZG+83TVDsogjUHETaWPOTl8UuE/3nOXjtNzWAnfH8DAmJ4bh5qJ/\neiIi0jEOnirjw80nOJlfhckEY+KCuWt0b/y8XY2OJiI3Qe8eRdqJs5OFqSMiLo7AZebx8c5clm89\nybrdZ7gjKYxJw8KMjigiIl3Y8XOVpG4+waHT5QCMjg9hWlIYwT3dDU4mIt+EiptIO3NxsjI1OYLb\nh/ZiQ8ZZ1u46w7ItFwvczHFRjIj210LeIiLSZs4WV7P0sxPsOVoCwOA+vtw7LorE2BCKi88bnE5E\nvikVN5EO4uJkZfrI3kwYGsqnmWdZszOXv39ymH9+epTRg4OYPFyfgop0NTU1NSxYsIDKykoaGxuZ\nN28eY8eOBeD999/n9ddf59NPPzU4pXQVRRV1LN9ygrQDhdiBvqHe3DuuDwPCfYyOJiJtQMVNpIO5\nOn9R4LJOlrF04zE2ZZ1jU9Y54qN6ckdSONHhPTCZtOCpiKNbunQpkZGRzJ8/n8LCQubOncuaNWso\nLS1l3bp1RseTLqL8fD0rt5/is+xzNLfYCQvw4N7xfYjt01O/S0S6EBU3EYO4Olu5a2wUI/r7k3mk\nmLW7c8k+Xkr28VLCAz2YMjyc4QMDtACqiAPz8fHh8OHDAFRVVeHjc3Hk4/e//z1PPPEETz75pJHx\nxMFV1zXycdppNmScpaGphUAfV2aN60NidABmFTaRLkfFTcRgZrOJxOgAEqMDOJZXySe7z5BxuIg3\nVh5kyebjTBwWyviEEC3mLeKApk+fTmpqKpMnT6bq/7d33/FRlXn//18zmfTeO6lAIBA6inQUBEEX\nYRVcEctadoVbf676U6M+9L5VtogF9Xatu2u7BZRQdtkVFAURUQiB0FtoSQippNdJ5vsHEIhAAE04\nM8n7+XjwyMycMyfvXCRzzWeu61ynvJy3336bH3/8EVdXV/r06WN0PHFQtfVWvtyYzRcbjlBT14i/\ntyu/GRbHVb3C9GGfSAemwk3EjiRG+pIY6UthaQ1fpefw7dajfL46i3+uO8SwlHDGDowixN/D6Jgi\ncpGWLl1KREQE77//Prt37+aJJ57Aw8ODN99886KP4e/vgcXi1CZ5goO92+Q4l4Oynq2+oZEv1h9i\n4aq9lFXW4+Ppwm+u7cF1V8Xi4nxxvyOO1K7gWHmVtX04UlZo37wq3ETsULCfO7dc05VfDYvl28w8\nvkzPZtWmHL7elEP/blV7kQoAACAASURBVMGMGxxNYqSvzl0QsXMZGRkMGzYMgKSkJLKysggLC+Oe\ne+4BoKCggIceeohXXnnlvMc4fry6TbIEB3s7zIqCytpSY1MT67YdY9m6g5SU1+Hm4sTkYXGMHRSN\nu6uFstKL+x1xpHYFx8qrrO3DkbJC2+RtrfBT4SZixzzcnBl/RReuGRhF+p4CVmzIZtPeQjbtLSQ+\nwodxg6IZ0D0YJ7OmxojYo5iYGDIzM7n22mvJzc0lPDycFStWNG8fM2ZMq0WbdG5NNhub9hSy+NsD\nHCupxtliZvzgLlw3JAYvd02fF+lsVLiJOACLk5kre4ZxRY9Q9maXsmJDNpn7i3hr6Q4CfdwYOzCK\n4X0icHfVn7SIPZk2bRqpqanMmDEDq9XKs88+a3QkcQA2m41tB0pI+zaLI/mVOJlNjOobwfVD4/D3\ndjU6nogYRO/yRByIyWSiexd/unfx51hJNV+mZ7Nuax7zv97P0nUHGdEngmsGRBPo62Z0VBEBPD09\nmTdv3nm36xpu8lN7s0tJW5PF3pwyTMCVPUP51fA4QnV+s0inp8JNxEGFBXhw27ju3Dg8ntWbc1m1\nKYcVG7L5cmMOA5OCuXZwF+LCfYyOKSIiF+HwsQoWrz3A1qxiAPomBnHjiHiiQ7wMTibSOdlsNmzY\nsNlsADQ137bR1GKbjSZsWExOQPsupKLCTcTBebk7M+mqWK4d3IUNu/JZseEIG3YVsGFXAd2ifBk3\nuAt9E4Mwm7WQiYiIvTlWUs2StQfYsKsAgKQufkwZmUBipK/ByaSjqLXWUVxbQlFNCcU1xdiONVJV\nVddceLT42uIxTn5twgat7NN0xrbT+2GztSx2ziiCbCe3cXLfU4UQJ/c5sQ2cLCYaGqyntpyd96z8\ntLLt1O1T+zSdvn+O/S+V2WTm6VEPEmIKb6v/urOocBPpIJwtZob2DueqXmHsPHyclRuy2XagmL05\n2wjxd2fswGiG9Q7H1aVtlhUXEZGfr6S8lmXrDvLd1mM02WzEhnkzdWQCPWP9tWKwXJImWxNldeUU\n1ZRQVHuiOCuqKaHo5NeKhkqjI14UEyZMJhNmTGAyYcKE2WwGmw0TZkwmEyY4+dXUvP9Pv5pN5hP7\nmU2t79t8rLOPDSbMp26fzHTq79LU4v7p265OroR7hdBY1X5tpMJNpIMxmUwkxwaQHBtAbmElX6Zn\n8/32fD75ci9L1h5gVL9IxvSP0gnuIiIGKK+u59/rD/N1Ri7WxibCAz2YMiKe/t2CVbDJef101Kzo\n5O2imhKKa0uwNlnPeo7ZZCbQzZ8o7wgC3QMIcgsgyD2QqOBgystqMZn4SSFzRjHSosjhnNuab58q\naDCfUezQorgxndyXM4szaFFMnYujXQ4gwMObwqr2y6vCTaQDiwz24o4JPbhxRALfZOTwdUYuy9cf\n5osfj3BFz1DGDYqmS6hjXdhSRMQRVddaWbHhCCvTs6mrbyTQx43Jw+MYkhymqezSyqjZiZGz842a\neVo8iPAMI9g98ERx5h5AkFsgQe4B+Ln64mQ+e5ZNcLA3hSbHKYbkNBVuIp2Ar6cLk4fHc92VMazf\ncYyVG7P5fvsxvt9+jB4x/lw7OJpe8YEnPw0TEZG2Ut/QyKqMHP69/jBVtVZ8PF349cgERvSJwNmi\na3B2JnWN9c3TFy9l1CzAzZ9Ir/ATRZl74Ml/AQS6BeDh7G7ATyJGUeEm0om4ODsxsm8kw/tEsP1A\nMSs2ZLPr8HF2HT5OeKAH4wZFMyQ5DBdnnQcnIvJLWBubWLs1j3+uO0hpZT0erhamjoznmgHROte4\ng2qyNVFeX0FhdfHZo2a1xVTUtz5qdrowu/ComXROF1W4zZkzh8zMTEwmE6mpqaSkpJy1z0svvcSW\nLVv46KOPLvo5ImIMs8lESkIQKQlBHMmvYOXGbH7cmc8HX+wh7dsDjD55HpyPp4vRUUVEHEqTzcaP\nO/NZsvYAhaW1uDibmTgkhvFXdMHTzdnoePIL1TXWU9y88MfpUbPS+lLyq4paHzXz16iZ/DIXLNw2\nbNjA4cOHWbBgAVlZWaSmprJgwYIW++zfv5+NGzfi7Ox80c8REfvQJdSbuyf1ZOrIBL7OyGH15lyW\nrTvEv384wlW9Qhk7qAuRQZ5GxxQRsWs2m40t+4tY/O0BcgqrcDKbuHpAFJOGxODrpcWgHMWpUbOi\nM4uzixg183LxbDlq5hZAoHsAwe6BGjWTNnPBwm39+vVcc801ACQkJFBWVkZlZSVeXqcvCPmnP/2J\nhx56iDfeeOOinyMi9sXf25WpIxOYNCSW77bl8eXGbL7NzOPbzDx6xwdy89huRPi5adUzEZGf2HX4\nOGlrssg6Wo7JBEN7h/GroXEE+WkkxR6db9Tsos41O2PU7MzFQGIiQhxq9UNxTBcs3IqKikhOTm6+\nHxAQQGFhYXMRlpaWxuDBg4mMjLzo54iI/XJ1ceLqAVGM7hfJlv1FrNxwhG0Hitn29nqigr24dnA0\nV/QMxeKkk+pFpHPLKazktbRtbNlbCMCA7sFMHh6vWQoGO3vU7IyvrYyaeVjcifAMPT2V8eSoWZB7\nIP4aNRM7cMmLk5y64jlAaWkpaWlp/P3vfyc/P/+innM+/v4eWCy//A8iONhxljZ3pKzgWHmVtW1c\nG+rDtUPj2XvkOEvXZPHd1qO8v3wXi9ceYOLQeCZcFYu3h/2eB2fPbftTyiriWI7kV/Dn/9tMTZ2V\n5LgApoyIJy7cx+hYnUaLUbPakhbFWaujZq5+RPp3PceoWQAezh4G/CQiF++ChVtISAhFRUXN9wsK\nCggODgbghx9+oKSkhFtvvZX6+nqOHDnCnDlzWn3O+Rw/Xv1zf4ZmjnSRPkfKCo6VV1nbnr+7hUdv\nG8ik/QWs2pTDmi1H+eg/u1jw1R6G9g5n3MBoQgPsq8NzlLaFzpdVhZ84urziKl5asIXaOisP3dKP\n3jH+RkfqcJpsTZRUl7K/9EiL88yKa4sprLnwqFngyRGz09c306iZOL4LFm5Dhw7l9ddfZ/r06ezY\nsYOQkJDmKY/jx49n/PjxAOTk5PDEE0+QmppKRkbGeZ8jIo4ryNedaWO6csPQONZmHuXL9By+ychl\ndUYufbsGMW5QNN2i/XQenIh0WIWlNcydv4WK6gZmju/OmIFdHOaDF6M1NjVS2VBFRX0lFQ2VJ76e\n+a+hkor6Cirqq6ior8BqazzrGGeOmjWPljUvoa9RM+nYLli49e/fn+TkZKZPn47JZOKZZ54hLS0N\nb29vxo4de9HPEZGOw93VwrjBXbh6YBSb9hSyYkM2m/cVsXlfEbFh3owbHM3A7iE6D05EOpTjFXW8\n+OlmjlfUMW1MIqP6Rl74SR1crbWOyoZKyk8WX5X1J283F2GVVDScKMSqGi48u8rZbMHbxZtIrwjC\n/YLxNvm0WEJfo2bSmV3UOW6PPPJIi/tJSUln7RMVFdV8DbdzPUdEOh4ns5nBPUIZlBTC/twyVm7I\nJmNvIe8s28nnPllcMyCaEX0i8HC75NNpRUTsSnlVPXPnb6aorJbJw+K4dnAXoyO1iyZbE1UN1WeM\nhFWcLLxO3T5RmFWe3F7f1HDBY3paPPBy8SLCMwwvFy+8nb3wcfHCy+XEV28XL7xOPubq5No8a8OR\nppGLXA56NyUiv5jJZKJrlB9do/woOF7Nl+k5fLc1j4Xf7GfpuoOMSIlg7MAoLY0tIg6puraBlxds\nIa+4mvGDu3D90FijI12S+sYGKuorKSsu5khRPhX1VSdGxhoqzpqqWFlfhY3WF5VzMjnh7eJFqGcI\n3s4nCq/mf833vfF28cTb2UsjZCJtRIWbiLSpEH8Pbh3bjcnD41iz5ShfpWfzZXo2X23KZkD3EK4d\nFE1CpK/RMUVELkptvZVXFmZypKCSUf0iuWl0guHn8dpsNqqtNS1GxMrrK5pHwU4VYadGxmob6y54\nTHeLG97OXoT4Bp0uvJw9zyjCvE7e98bdomt6ihhBhZuItAtPN2euuzKGcYOi2birgBUbjpC+u4D0\n3QUkRvoyblA0/bsFYzar8xcR+1Tf0Mhrn28l62g5Q5LDmDGuW7sVLNYmK5UnC7BTi3O0XLSjZVHW\nZGtq9XhmkxlvZ08C3QNOjoJ5E+rrj6XR9eR0RU98ThZkXi5eOJv1llDE3umvVETalcXJzJBeYVyZ\nHMruI6Ws3HCEzKxi9ueWEeTrxthB0QxPCcfNRS9HImI/rI1NvLlkO7uPlDKgWzB3TUzCfAlFm81m\no7ax9mSxVXXO88PK6yubF/aosdZc8JiuTi54O3sR4x19numJp/95WNwxm1ouEKVzxkQcm94pichl\nYTKZ6BHjT48Yf/KKq/hyYzbrth/j06/2sWTtQUb1jeDqAVEE+LgZHVVEOrmmJhvv/nMnW7OK6RUf\nwL03JONkblkE1VhrOFB2mMqSMvJKis85Knaui0CfyYQJL2dP/F19ifaOPLFQxzmKsFOPuTi5tOeP\nLSJ2ToWbiFx24YGezByfxOQR8azenMvXm3L4z49HWLkxm0E9Qrh2UBdiwnSRZhG5/JpsNv7+n11s\n3F1At2g/Zt3YG2eLmbK6CrLKDrK/9CBZpQfJrcw75yIep5ezD2+xeqK3ixc+zqdWUjwxRdHT2eOs\nUTERkfNR4SYihvHxcOGGoXFMuKILP+zIZ+XGbH7Ykc8PO/JJ6uLHuEFdSEkMvKTpSSIiP5fNZuPT\nL/exblse0dFmhgxvYOH+RWSVHqSwprh5P4vZQoJfLIm+cfSK6oqt1tI8Kubq5KKFO0SkXahwExHD\nOVucGN4ngmEp4ew4WMKKjdnsOFjC7iOlhAZ4MG5QNFf1CsPVWUtKi0jba7I1cbTyGIvSN7KzLAuP\n/qUUWWpZuP/EdneLG8mBSST6xpHgF0cXn6jmxTx03piIXC4q3ETEbphMJnrFB9IrPpCcgsoTI3A7\nj/HRij0s/vYAo/pFcnX/SHy9XI2OKiIOrKHJypHyHLJKD7K/7CAHyg5RY60FwBIIXs7edPXvRoJf\nHIm+cUR4hWlKo4gYToWbiNilqBAv7prYg6kj41mVkcvqzbn86/tDfPHjYa7sGca4QdFEhXgZHVOk\nVVVVVTz22GOUlZXR0NDArFmzqK+v55133sHZ2ZmAgABefPFFXF31YUR7qrHWcrDscHOhdqg8u8XC\nIZ5mP6yFgXhYQ5g1diQJweGa7igidkeFm4jYNV8vV6aMiGfikBi+336MlRuz+W5bHt9tyyM5LoBr\nB0WTHBegN1lilxYvXkxcXBwPP/ww+fn53H777YSGhvLee+/h7e3NE088wcqVK7n++uuNjtqhlNdX\nkFV6qLlQy6k42ryQiAkTkV7hJPqdmPZYlOvO/BXZ+Hq68PiM/oT6exicXkTk3FS4iYhDcHV2YnS/\nSEb2jWDr/mJWbjzCjoMl7DhYQmSQJ+MGRXNlchjOFk1nEvvh7+/Pnj17ACgvL8ff358PPvgAAKvV\nSmFhIaGhoUZGdHg2m43i2pLm1R73lx2koLqoebvF5ES8b8yJaY9+ccT7xuBucQdgw658FqzcgZe7\nM49M76uiTUTsmgo3EXEoZpOJvl2D6Ns1iEPHylm5MZuNuwr4+392s+jbA4zpH8nofpEEGx1UBJg4\ncSJpaWmMHTuW8vJy3n77bQDS0tJ47bXXGDNmDIMHDzY4pWNpsjWRV5V/ulArPUhZfXnzdjcnN3oG\ndG8u1GK8o3B2cj7rOFv2F/HuP3fi5uLEH6b1ITJYU69FxL6ZbDbb2RchMUBbrMjkSCs7OVJWcKy8\nyto+7DlrSXktqzblsHrLUWrqrLhYzNx7Y2/6JwQaHe2i2HPb/lRbZA0O7jzX6Fu6dCnp6ek899xz\n7N69m9TUVNLS0oATI26PPfYYo0aNanWqpNXaiMXSeVdUtTZayTp+mF2F+9lduJ89RVlUNdQ0b/d1\n86FHUCJJwQn0CO5KjG8kZnPrI++Zewv57/d/wGw28d/3DCE53jFeK0Skc9OIm4g4vAAfN24ancj1\nQ2NZuzWPf647xBufZTJuUDQ3j07EbNb5b2KMjIwMhg0bBkBSUhI5OTmsXr2aUaNGYbFYuPrqq9mw\nYUOrhdvx49VtksVRPiCotdZSYipk0+GdZJUe5FD5ERrOWEgkyD2Q3oHJJ0fUYgl2Dzp9jqsViour\nWj3+/twyXpq/BZvNxuwpKYR4u/yidnGUdgXHygqOlVdZ24cjZYX2/3BThZuIdBhuLhbGDoymT0Ig\n/7tkOys3ZnOspJr7bkjG3VUvd3L5xcTEkJmZybXXXktubi7e3t4888wzLFy4kNDQULZu3UpcXJzR\nMQ1VUV/ZfG5aVulBcirzaLI1AScWEonwCjuxkIhvHAl+sfi5+v7s73X4WAWvLMykwdrErBt7kRwb\n0FY/hohIu9M7GRHpcEL8PXjxv0bw/Ps/sDWrmDkfb+LBqSkE+bkbHU06mWnTppGamsqMGTOwWq08\n//zz1NfXM2vWLFxcXAgKCuLBBx80OuZlY7PZKKk9fuL8tLIT56flVxc2b3cyORHrE01KRBLhzhHE\n+8bi4dw2f7dHi6p4acEWauus3HNDT/p105mwIuJYVLiJSIfk6e7MgzelMH/VflZtyuG5D9OZPaU3\nXaP8jI4mnYinpyfz5s076/GRI0cakObya7I1cayqoEWhVlpX1rzd1cmFHgHdSPA9Me0xxqcLLk7O\nbT49qqC0hrnzN1NZ08AdE5K4smdYmx1bRORyUeEmIh2Wk9nMrWO7ERHowSdf7uPFTzdzx4QkruoV\nbnQ0kQ7J2mQluyK3uVDLKj1EtfX0QiJezp70De514vw03zgivcJxMrfvwisl5bXM/XQzpZX1TL+6\nKyP6RLTr9xMRaS8q3ESkwxvdP4qQAA/+ung77/1rF3nF1dw4Ih6zLtot8ovUWus4VH6keWn+g+VH\naGhqaN4e6BZA76CeJPjFkugbR4hH8OmFRC6D8qp65s7fQlFZLZOHxzFuUPRl+94iIm1NhZuIdArJ\nsQE8OXMA8z7fyvL1h8krruaeST1xdem8y6yLXKrK+qrmKY9ZpYfIrsxtsZBIuGfoiYVETl5D7Zcs\nJPJLVdU28NKCLRwrqWbCFV24/qpYw7KIiLQFFW4i0mmEB3ry1MyBvLl4Gxl7C/njJ5t4YGoKAT5u\nRkcTsUvFNcfPKNQOcqy6oHmbk8mJGO/ok4VaLAm+sXg4exiY9rSaOiuvLMwku6CS0f0j+fWohMs6\n0ici0h5UuIlIp+Ll7swfpvXl45V7+DYzj+c+TOeBqSnEhfsYHU3EUKcWEjlzRO14XWnzdhcnF5L8\nuzaPqMX6ROPi5GJg4nOrb2jk9UVbOXC0nKt6hXHr2G4q2kSkQ1DhJiKdjsXJzO3jk4gI9GTBN/v5\n0ycZ3D2pJ4OSQoyOJnLZNDY1cqQit7lQO1B2iKqG0xf79nL2pE9QcvO0xyiviHZfSOSXsjY28eaS\n7ew+UsqA7sHceV2SzmUVkQ5DhZuIdEomk4lxg7sQGuDB28t28Ncl28kbHsf1V8Xq03npkGqtdewu\n2XfyYteHOFR2mPozFhIJcPOnZ0ASiX6xJPrFEeoR4lB/C41NTbyzbAdbs4rpHR/IfTck42Q2Gx1L\nRKTNqHATkU6tT2IQqbcNYN5nW1my9iB5xdXcOSEJF2f7HlkQuVi11jre3fYh+0qzaDy5kAhAuGdo\n87L8iX5x+Ls57jUOm2w2/vHv3aTvKaR7tB+zbuyFxUlFm4h0LCrcRKTTiwr24unbB/JG2jZ+3JlP\nYWkN/zWlN75erkZHE/nFrDYrJXXHiffvQhevLiT6xhHvF4uXs6fR0dqEzWbj/77cy7rtx4gL9+GB\nX6fogxcR6ZBUuImIAD6eLjx6S1/+8Z89rN9xrHnRki6h3kZHE/lFvJw9eebK/5/gYG8KCyuMjtOm\nbDYbn6/J4uuMXKKCvXjo5j64u+qtjYh0TJpHICJykrPFibsn9WDqyHhKyuv448cZbN5XaHQsETmP\nf60/zH9+OEJogAcPT++Ll7uz0ZFERNqNCjcRkTOYTCYmDoll1o29sGHjjUXb+M8Ph7HZbEZHE5Ez\nfLkxm8XfHiDQx41Hp/fF19P+Lk0gItKWVLiJiJzDgO4hPHHrAPy8XflsdRZ///durI1NF36iiLS7\ntZlH+XTVPny9TkxxDvBxMzqSiEi7U+EmInIeMWHePDVzILFh3ny3LY+5n26morre6FgindqPO/P5\nx3924+XuzCPT+xHi72F0JBGRy0KFm4hIK/y9XXns1v4MSgphb04Zz3+YTm5RldGxRDqlLfuKeO9f\nO3FzdeLhaX2JDOoYK2OKiFwMFW4iIhfg6uzEfb9K5oahsRSW1jLno3S2HSg2OpZIp7LzUAlvLtmO\nk5OJh27qS0yYVnwVkc5FhZuIyEUwm0xMHh7PvTf0pMFq49XPMvkqPVuLlohcBvtySnlt0VYAHpia\nQmKUr8GJREQuPxVuIiKX4MqeYTx2az+8PVz4v6/28fHKvVq0RKQdHT5WwaufZdLYaOP+yb3oGRtg\ndCQREUOocBMRuUQJEb48PXMg0SFefLM5l1c/y6SqtsHoWCIdTm5RFS8t2EJtXSN3T+pJ365BRkcS\nETGMCjcRkZ8h0NeNJ2b0p29iEDsPHef5DzeRX1JtdCyRDqPgeDVz52+msqaB2yckcUXPUKMjiYgY\nSoWbiMjP5OZiYfaU3oy/ogv5JdU8/2E6uw4fNzqWiMMrPF7Di59uoayynluu7sqIPhFGRxIRMZwK\nNxGRX8BsNnHz6ETuvC6J2vpGXl6whTVbco2OJeKwyqrqefrtdRSX13LjiHjGDoo2OpKIiF1Q4SYi\n0gaGp0TwyPS+uLta+OCLPcxftY+mJq04KXIpKmsaeGn+FnILq5hwZRcmDYkxOpKIiN1Q4SYi0ka6\nd/HnqdsHEh7owcqN2by2aCs1dVajY4k4hJo6K68szCSnsJKJQ+P49cgETCaT0bFEROyGCjcRkTYU\n4ufOk7cNpFdcAFuzipnz8SaKSmuMjiVi1+oaGnnt860czCtnaK8w7p3cW0WbiMhPWIwOICLS0Xi4\nWXjwphTmr9rPqk05PPdhOrOn9KZrlJ/R0eQyq6qq4rHHHqOsrIyGhgZmzZpFcHAw//M//4PZbMbH\nx4eXXnoJd3d3o6MaxtrYxJuLt7Mnu5SB3YO547okzGYVbSIiP6URNxGRduBkNnPr2G7cNq4bVTVW\nXvx0M99vzzM6llxmixcvJi4ujo8++oh58+bxwgsv8Pzzz/P444/z8ccfExMTQ1pamtExDdPY1MTb\ny3aw7UAxKQmB3HtDMk5mvTURETkXvTqKiLSj0f2jeGhaH1wsTrz3r10sWpNFk02LlnQW/v7+lJaW\nAlBeXo6/vz9vvfUWKSkpAAQEBDRv72yabDb+/u/dbNpTSFIXP+6f3AuLk96WiIicj14hRUTaWXJs\nAE/OHECIvzvL1x/mzcXbqatvNDqWXAYTJ07k6NGjjB07lhkzZvDYY4/h5eUFQHV1NUuXLmX8+PEG\np7z8bDYbn6zcy/fbjxEf4cN/TU3BxdnJ6FgiInZN57iJiFwG4YGePDVzIG8u3kbG3kL++MkmHpia\nQoCPm9HRpB0tXbqUiIgI3n//fXbv3k1qaippaWlUV1fz+9//nrvuuouEhIRWj+Hv74HF0jZFTXCw\nd5sc55ew2Wx8sHwn32zOJS7Chxd+PxQvD5ez9rOHrBdLWduPI+VV1vbhSFmhffOqcBMRuUy83J35\nw7S+fLxyL99mHuW5D9N5YGqKw3VKcvEyMjIYNmwYAElJSRQUFFBfX8/999/PpEmTmDJlygWPcfx4\ndZtkCQ72prCwok2O9Uv8c91BFq89SFiABw9OTaGmqo6aqroW+9hL1ouhrO3HkfIqa/twpKzQNnlb\ne0+gqZIiIpeRxcnM7eO7M/3qrpRX1fOnTzJYuyXX6FjSTmJiYsjMzAQgNzcXT09P3n//fQYPHsxN\nN91kcLrLb+XGbBavPUiQrxuPTO+Lj+fZI20iInJuGnETEbnMTCYT4wZFE+rvztvLdvCXj9KZPCyO\n64fG6tpVHcy0adNITU1lxowZWK1Wnn32WR599FGioqJYv349AFdccQWzZ882OGn7+zbzKPNX7cPP\ny4VHbumnacIiIpfoogq3OXPmkJmZiclkIjU1tXk1LICFCxfy+eefYzabSUpK4plnnmHDhg08+OCD\ndO3aFYBu3brx9NNPt89PICLioPokBpF62wDeSNvGku8OkldSzZ0TkrRIQwfi6enJvHnzWjz23Xff\nGZTGOD/sPMYH/9mNl7szj0zvR4hf571unYjIz3XBwm3Dhg0cPnyYBQsWkJWVRWpqKgsWLACgpqaG\n5cuX88knn+Ds7MzMmTPZvHkzAIMHD+a1115r3/QiIg4uKtiLlx4cyX+/u54fd+ZTWFrDf03pja+X\nq9HRRNrE5n2FvPfPXbi5Wnh4Wl8igjyNjiQi4pAueI7b+vXrueaaawBISEigrKyMyspKANzd3fng\ngw9wdnampqaGyspKgoOD2zexiEgH4+ftyqO39GNIchgHjpbz3IfpHMl3nJOxRc5nx6ES/rpkO84W\nMw/d3IeYMC3EIyLyc12wcCsqKsLf37/5fkBAAIWFhS32eeeddxg7dizjx48nOjoagP379/O73/2O\nW265hXXr1rVxbBGRjsXZYubuST2YOjKekvI6/vhxBpv3FV74iSJ2am92Ka8v2gqYeGBqbxIjfY2O\nJCLi0C55cRKbzXbWY/feey8zZ87knnvuYcCAAcTGxjJ79mwmTJhAdnY2M2fOZOXKlbi4nH/1qLa6\nTo0jLavtSFnBsfIqa/twpKzgWHlPZb3jht50iw3k5U8zeCNtG7df15MpoxPtatESR2pXMcahY+XM\n+zyTxkYbs6b0pkdsgNGRREQc3gULt5CQEIqKiprvFxQUNE+HLC0tZd++fQwaNAg3NzdGjBhBRkYG\nAwYM4LrrrgOgw0WLcQAAEh1JREFUS5cuBAUFkZ+f3zwady5tcZ0aR7rWgyNlBcfKq6ztw5GygmPl\n/WnWruHePP6b/ry2aCv/WL6TfUdKmHltEs4W46/g0t7XqBHHl1tYycsLMqmtb+S+G5LpmxhkdCQR\nkQ7hgu8Chg4dyooVKwDYsWMHISEheHl5AWC1Wnn88cepqqoCYNu2bcTFxbFs2TLef/99AAoLCyku\nLiY0NLS9fgYRkQ4nJsybp2YOJDbMm3XbjvHS/M1UVNcbHUukVfnHq5k7fwuVNQ3cMSGJwT3U94uI\ntJULjrj179+f5ORkpk+fjslk4plnniEtLQ1vb2/Gjh3LrFmzmDlzJhaLhe7du3P11VdTVVXFI488\nwqpVq2hoaODZZ59tdZqkiIiczd/blcdu7c/flu9i4+4CnvsgnQdv6kOkVuUTO1RSXsvcTzdTVlXP\nb67pyvCUCKMjiYh0KBd1jtsjjzzS4n5SUlLz7SlTpjBlypQW2728vHjrrbfaIJ6ISOfm6uzEfb9K\nJjzQg2XrDjHno3R+96te9I4PNDqaSLOyqnpenL+F4vI6po6M55qB5z81QkREfh7jT5gQEZFWmU0m\nJg+P594betJgtfHqZ5l8lZ59zsWiRC63ypoGXpq/mfySaiYOiWHikFijI4mIdEgq3EREHMSVPcN4\n7NZ+eHu48H9f7eOjlXuxNjYZHUs6sZo6K68s3EJOYRVXD4hiyoh4oyOJiHRYKtxERBxIQoQvT88c\nSHSIF6s35/LKwkyqahuMjiWdUF1DI/M+38rBvAqG9g7jlmu62tVlK0REOhoVbiIiDibQ140nZvSn\nb2IQuw4f5/kPN5Ff8ssvqSJysRqsTfzv4m3szS5lUFIId07ogVlFm4hIu1LhJiLigNxcLMye0pvx\nV3Qhv6Sa5z9MZ9fh40bHkk6gsamJt5ftYPuBElISArnn+p6YzSraRETamwo3EREHZTabuHl0Inde\nl0RtfSMvL9jC6i25RseSDqzJZuNvy3eTsbeQpC5+3D+5FxYnvZUQkY5t9epVF7XfCy+8wNGj7dcP\n69VWRMTBDU+J4NFb+uHuauHDL/bw6Vf7aGrSipPStmw2Gx+v3Mv6HcdIiPDhgV+n4OLsZHQsEZF2\nlZd3lK++WnFR+z755JNERES2W5aLuo6biIjYt27Rfjx1+0DmfZbJl+nZ5B+v5r4bknF31cu8/HI2\nm43Pvsli9eZcuoR48f/d3Ac3F/1uiUjH9/LLf2bXrh0MHz6IceMmkJd3lFdffZM//vF/KCwsoKam\nhrvuupehQ4dz2223MXv2H/jmm1VUVVVy5MhhcnNzeOCBhxkyZOgvzqJXXRGRDiLEz50nbxvIW0u3\nszWrmDkfb+LBqSkE+bkbHU0c3D/XHeKLDUcID/TgD9P74unmbHQkEelkFn69n427C9r0mIOSQrh5\nTGKr+9xyy22kpS0kLi6BI0cO8eab73H8eAmDB1/JhAmTyM3N4emnH2fo0OEtnldQkM/cua/xww/f\ns3TpIhVuIiLSkoebhQdvSmHBqv18tSmH5z5MZ/aU3nSN8jM6mjiolRuOsOS7gwT5uvHI9H74eLgY\nHUlExBA9eiQD4O3tw65dO1i2LA2TyUx5edlZ+6ak9AUgJCSEysrKNvn+KtxERDoYJ7OZ34ztRniQ\nJ5+s3MuLn27mjglJXNUr3Oho4mDWbMll/tf78fd25dFb+uHv7Wp0JBHppG4ek3jB0bH25ux8YrbB\nl19+QXl5Of/7v+9RXl7O3Xffdta+Tk6nzwG22drmvHMtTiIi0kGN7hfJQ9P64GJx4r1/7WLRmiya\n2qjzkI7vhx3H+PCLPXh7OPPI9L4Ea8qtiHRCZrOZxsbGFo+VlpYSHh6B2WxmzZqvaWhouDxZLst3\nERERQyTHBvDkzAGE+LuzfP1h3ly8nbr6xgs/UTq1jL2FvPevXbi7Wnh4Wl/CAz2NjiQiYoiYmDj2\n7NlNVdXp6Y6jRo3h++/X8uCDv8fd3Z2QkBD+/vd32z2LpkqKiHRw4YGePDVzIG8u3kbG3kL++Mkm\nHpiaQoCPm9HRxA7tOFjCW0u342wx89DNfegS6m10JBERw/j7+5OWtrzFY+HhEXzwwfzm++PGTQAg\nONibwsIK4uNPT+mMj0/kjTfeaZMsGnETEekEvNyd+cO0vozoE8GR/Eqe+yCdg3nlRscSO7M3u5TX\nF23FZDLxwK9TSIj0NTqSiIicpMJNRKSTsDiZuX18d6Zf3ZXy6nr+9EkGG3blGx1L7MTBvHJe/SyT\nxiYb90/uRY8Yf6MjiYjIGVS4iYh0IiaTiXGDonlgagpOZhNvLd3Bsu8OttmKV+KYcgoreXnBFuoa\nGrn3hmT6JAYZHUlERH5ChZuISCfUJzGI1NsGEOTrxpLvDvL2sh3UN2jRks4ov6Sal+ZvoarWyp0T\nejAoKcToSCIicg4q3EREOqmoYC+emjmQxChfNuwq4C+fbqasss7oWB1KVVUVs2fP5rbbbmP69Oms\nXbuWpqYm5s6dy5VXXml0PIrLapk7fzNlVfXcOrYbw1J0rT8REXulwk1EpBPz8XTh0en9GJIcxoGj\n5Tz3YTpH8iuMjtVhLF68mLi4OD766CPmzZvHCy+8wDvvvEN4eLjh01PLKut4cf5misvrmDoynqsH\nRBmaR0REWqfCTUSkk3O2mLl7Ug+mjoynpLyOP36cweZ9hUbH6hD8/f0pLS0FoLy8HH9/f2bMmMGt\nt95qaK7KmgbmLthCwfEaJg6JYeKQWEPziIg4sl//+nqqq6vb/fvoOm4iIoLJZGLikFjCAjx49187\neWPRNn49KoHxV3TBZDIZHc9hTZw4kbS0NMaOHUt5eTlvv/02Xl5el3QMf38PLBanNskTHOxNdW0D\ncz7JILewiuuHx3PPr3rZ5f9xcLDjXD9OWduPI+VV1vbhCFmdnMwEBZ14bW/PvCrcRESk2YDuIQT5\nuvPaoq18tjqLo8VVzLw2CWeLJmj8HEuXLiUiIoL333+f3bt3k5qaSlpa2iUd4/jxtvkUNzjYm5yj\npbyyYAv7c8oYlhLOr66Koaiosk2O35ZOXcTWEShr+3GkvMraPozOetddtzJnzkuEhYVx7FgeTzzx\nMMHBIdTU1FBbW8tDDz1Kz569aGxsoqioEk9Pz1+ct7XCT4WbiIi0EBPmzVMzB/L6oq2s23aMwuM1\nzJrSG28PF6OjOZyMjAyGDRsGQFJSEgUFBTQ2NuLk1DYjaJeiwdrIG2nb2JtTxuAeIdwxPgmzHY60\niYicS9r+f7G5YFubHrNfSG+mJE467/YRI0azbt23TJ16M2vXrmHEiNEkJHRlxIhRbNq0kU8++YAX\nXnixTTO1Rh+hiojIWfy9XXns1v4MSgphb04Zz32QTm5RldGxHE5MTAyZmZkA5Obm4unpaUjR1tjU\nxIsfb2LHwRL6JgZx96SemM0q2kREWnOicFsLwHffrWHYsJGsWbOK3//+t/z1r69TVlZ2WfNoxE1E\nRM7J1dmJ+36VTHigB8vWHWLOR+mk3jGYSH93o6M5jGnTppGamsqMGTOwWq08++yzPPfcc+zdu5fK\nykpuu+02xowZw5133tluGWw2G39bvov1O/LpEePP7ycnY3HS57Yi4limJE5qdXSsPcTHJ1BcXEh+\n/jEqKipYu3Y1QUEhPP30c+zevZM33nj1suZR4SYiIudlNpmYPDyesEAP/rZ8N5+u3MMj0/oaHcth\neHp6Mm/evBaPDRky5LJmOF5Rd6Joiw3gv6b0wrmNFjoREekMhgwZxjvvvMnw4SMpLT1OQkJXANas\n+Qar1XpZs6hwExGRC7qyZxgJEb4nVs2yNhodRy5BgI8bT98+kD49wigvbf/lqkVEOpKRI0fzu9/d\nxT/+8Sm1tTU8//wzfPPNV0ydejNffbWS5cuXXbYsKtxEROSiBPu5E+zv4TCrkclpceE+uDprpE1E\n5FL16JHMmjU/Nt//5JPPm28PGzYSgIkTb7gsWTTJXURERERExM6pcBMREREREbFzKtxERERERETs\nnAo3ERERERERO6fCTURERERExM6pcBMREREREbFzKtxERERERETsnAo3ERERERERO6fCTURERERE\nxM6pcBMREREREbFzJpvNZjM6hIiIiIiIiJyfRtxERERERETsnAo3ERERERERO6fCTURERERExM6p\ncBMREREREbFzKtxERERERETsnAo3ERERERERO2cxOsDPMWfOHDIzMzGZTKSmppKSktK87fvvv+fl\nl1/GycmJESNGMGvWLAOTntBa3jFjxhAWFoaTkxMAc+fOJTQ01Kio7N27l/vvv5877riDGTNmtNhm\nj23bWl57a9u//OUvbNq0CavVyn333ce4ceOat9lb27aW1Z7ataamhscff5zi4mLq6uq4//77GT16\ndPN2e2rXC2W1p3Y9pba2lkmTJnH//fczZcqU5sftqV3lbI7URzpS/wiO1Ueqf2wf6h/bh/rIi2Rz\nMD/++KPt3nvvtdlsNtv+/fttN998c4vtEyZMsB09etTW2Nhou+WWW2z79u0zImazC+UdPXq0rbKy\n0ohoZ6mqqrLNmDHD9tRTT9k++uijs7bbW9teKK89te369ettd999t81ms9lKSkpsI0eObLHdntr2\nQlntqV2XL19ue+edd2w2m82Wk5NjGzduXIvt9tSuF8pqT+16yssvv2ybMmWKbdGiRS0et6d2lZYc\nqY90pP7RZnOsPlL9Y/tQ/9h+1EdeHIebKrl+/XquueYaABISEigrK6OyshKA7OxsfH19CQ8Px2w2\nM3LkSNavX29k3Fbz2hsXFxfeffddQkJCztpmj23bWl57M2jQIObNmweAj48PNTU1NDY2AvbXtq1l\ntTfXXXcd99xzDwB5eXktPn2zt3ZtLas9ysrKYv/+/YwaNarF4/bWrtKSI/WRjtQ/gmP1keofL39W\ne+NI/SOoj7xYDjdVsqioiOTk5Ob7AQEBFBYW4uXlRWFhIQEBAS22ZWdnGxGzWWt5T3nmmWfIzc1l\nwIABPPzww5hMJiOiYrFYsFjO/Sthj23bWt5T7KVtnZyc8PDwAODzzz9nxIgRzcP99ta2rWU9xV7a\n9ZTp06dz7Ngx3nrrrebH7K1dTzlX1lPsqV3//Oc/8/TTT7NkyZIWj9tru8oJjtRHOlL/CI7VR6p/\nbB/qH9uf+sjWOVzh9lM2m83oCJfkp3kfeOABhg8fjq+vL7NmzWLFihWMHz/eoHQdiz227VdffcXn\nn3/O3/72N0NzXIzzZbXHdp0/fz67du3i0UcfZdmyZYZ3lK05X1Z7atclS5bQt29foqOjDfn+0nYc\nqY9U/3j52GPbqn9sH47UP4L6yAtxuKmSISEhFBUVNd8vKCggODj4nNvy8/MNnybQWl6AyZMnExgY\niMViYcSIEezdu9eImBdkj217IfbWtmvXruWtt97i3Xffxdvbu/lxe2zb82UF+2rX7du3k5eXB0CP\nHj1obGykpKQEsL92bS0r2Fe7rl69mlWrVnHzzTfz2Wef8eabb/L9998D9teu0pIj9ZEdpX8E+2vb\nC7G3tlX/2PYcqX8E9ZEXy+EKt6FDh7JixQoAduzYQUhISPO0iqioKCorK8nJycFqtfLNN98wdOhQ\nI+O2mreiooLf/va31NfXA7Bx40a6du1qWNbW2GPbtsbe2raiooK//OUvvP322/j5+bXYZm9t21pW\ne2vX9PT05k88i4qKqK6uxt/fH7C/dm0tq72166uvvsqiRYtYuHAhN910E/fffz9XXXUVYH/tKi05\nUh/ZUfpHsL+2bY29ta36x/bhSP0jqI+8WCabI82jOGnu3Lmkp6djMpl45pln2LlzJ97e3owdO5aN\nGzcyd+5cAMaNG8dvf/tbg9O2nveDDz5gyZIluLq60rNnT55++mnDhrG3b9/On//8Z3Jzc7FYLISG\nhjJmzBiioqLssm0vlNee2nbBggW8/vrrxMXFNT92xRVX0L17d7tr2wtltad2ra2t5cknnyQvL4/a\n2lpmz55NaWmpXb4eXCirPbXrmV5//XUiIyMB7LJd5WyO1Ec6Sv8IjtVHqn9sH+ofjctrT217psvd\nRzpk4SYiIiIiItKZONxUSRERERERkc5GhZuIiIiIiIidU+EmIiIiIiJi51S4iYiIiIiI2DkVbiIi\nIiIiInZOhZuIiIiIiIidU+EmIiIiIiJi51S4iYiIiIiI2Ln/B2i5qk9bBLR2AAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7fe34c38fb38>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "rFYs9dH3ui66",
        "colab_type": "code",
        "outputId": "89923358-2da3-4a32-f504-986136cc134b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "# Test performance\n",
        "trainer.run_test_loop()\n",
        "print(\"Test loss: {0:.2f}\".format(trainer.train_state['test_loss']))\n",
        "print(\"Test Accuracy: {0:.1f}%\".format(trainer.train_state['test_acc']))"
      ],
      "execution_count": 89,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test loss: 0.46\n",
            "Test Accuracy: 83.8%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "vKDmGXAyukpV",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Save all results\n",
        "trainer.save_train_state()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "P4yiDxOavmJy",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "So you can see that using GloVe embeddings but not freezing them resulted in the best results on the test dataset. Different tasks will yield different results so you need to choose whether or not to freeze your embeddings based on empirical results."
      ]
    },
    {
      "metadata": {
        "id": "F0yKVPF8xBQK",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# TODO"
      ]
    },
    {
      "metadata": {
        "id": "A0e8p2LkxD6x",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "* when to use skip-gram/CBOW\n",
        "    * According to Mikolov:\n",
        "        \n",
        "        Skip-gram: works well with small amount of the training data, represents well even rare words or phrases.\n",
        "        \n",
        "        CBOW: several times faster to train than the skip-gram, slightly better accuracy for the frequent words\n",
        "* Loading word2vec\n",
        "* interpretable conv filters since we apply it on words here\n",
        "* [contextualized word embeddings](https://arxiv.org/abs/1607.00578)"
      ]
    }
  ]
}